2024-05-09T19:55:34.644 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T19:55:34.645 DEBUG:teuthology.run:Teuthology command: teuthology --name suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi --description rgw:verify/{0-install accounts$/{main} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} --verbose --archive /home/teuthworker/archive/suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi/7699763 --owner scheduled_suriarte@teuthology -- /home/teuthworker/archive/suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi/7699763/orig.config.yaml 2024-05-09T19:55:34.690 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T19:55:34.778 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi/7699763 branch: wip-rgw-d4n-long-descriptions description: rgw:verify/{0-install accounts$/{main} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind} email: null first_in_suite: false job_id: '7699763' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rgw-d4n-long-descriptions ceph: conf: client: debug rgw: 20 rgw bucket counters cache: true rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw max chunk size: 4194304 rgw obj stripe size: 6291456 rgw s3 auth use sts: true rgw sts key: abcdefghijklmnop rgw torrent flag: true rgw user counters cache: true setgroup: ceph setuser: ceph global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled - \(OSD_SLOW_PING_TIME sha1: 04969094632d400ff0930fc029b1d19b9c187eef ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 04969094632d400ff0930fc029b1d19b9c187eef rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 compression type: random ec-data-pool: true frontend: beast storage classes: LUKEWARM, FROZEN s3tests: accounts: iam alt root: RGW99999999999999999 iam root: RGW88888888888888888 s3 main: RGW11111111111111111 force-branch: ceph-master slow_backend: true storage classes: LUKEWARM, FROZEN thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rgw-d4n-long-descriptions sha1: 86872c6b9201173c508ab909aaa65bdb12239dd6 owner: scheduled_suriarte@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 04969094632d400ff0930fc029b1d19b9c187eef sleep_before_teardown: 0 suite: rgw:verify suite_branch: wip-rgw-d4n-long-descriptions suite_path: /home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa suite_relpath: qa suite_repo: https://github.com/samarahu/ceph.git suite_sha1: 86872c6b9201173c508ab909aaa65bdb12239dd6 targets: smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIbTy3raqO4khV5rOqqDzh8MX5hOs+O6By/hpYAr6mpnsxb3z2nBAgcPVoX0Ayw25ENKd1f1+vX5V6ge8RfGkJM= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEOx++WTTmCNruefhchnjFlI04DbsUCT5C2bYVCLZkan++Vwy5oJe+5YugHaHyuTg9gdefdhRvRYvBmK0rpKMk= tasks: - install: extra_system_packages: deb: - s3cmd - maven rpm: - s3cmd - maven - ceph: null - openssl_keys: null - rgw: client.0: null - tox: - client.0 - workunit: clients: client.0: - rgw/run-bucket-check.sh - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - cls/test_cls_rgw_gc.sh - cls/test_cls_rgw_stats.sh - cls/test_cls_cmpomap.sh - cls/test_cls_2pc_queue.sh - cls/test_cls_user.sh - rgw/test_rgw_gc_log.sh - rgw/test_rgw_obj.sh - rgw/test_librgw_file.sh - rgw/test_awssdkv4_sig.sh - workunit: clients: client.0: - rgw/test_rgw_s3_mp_reupload.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - workunit: clients: client.0: - rgw/run-reshard.sh - s3tests-java: client.0: force-branch: ceph-master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: rgw_server: client.0 - workunit: clients: client.0: - rgw/run-versioning.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-09_19:24:12 tube: smithi user: suriarte verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T19:55:34.778 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa; will attempt to use it 2024-05-09T19:55:34.779 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks 2024-05-09T19:55:34.779 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T19:55:34.781 INFO:teuthology.task.internal:Checking packages... 2024-05-09T19:55:34.808 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '04969094632d400ff0930fc029b1d19b9c187eef' 2024-05-09T19:55:34.808 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T19:55:34.808 INFO:teuthology.packaging:ref: None 2024-05-09T19:55:34.808 INFO:teuthology.packaging:tag: None 2024-05-09T19:55:34.808 INFO:teuthology.packaging:branch: wip-rgw-d4n-long-descriptions 2024-05-09T19:55:34.808 INFO:teuthology.packaging:sha1: 04969094632d400ff0930fc029b1d19b9c187eef 2024-05-09T19:55:34.808 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rgw-d4n-long-descriptions 2024-05-09T19:55:34.949 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3307-g04969094-1jammy 2024-05-09T19:55:34.950 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T19:55:34.957 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T19:55:34.957 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T19:55:35.004 INFO:teuthology.task.internal:Saving configuration 2024-05-09T19:55:35.021 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T19:55:35.043 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T19:55:35.067 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi/7699763', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 19:49:55.450684', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK2RTPvx5IbDPH7jTxSrUQa6TSnPzrCSji8gumJ4Ii3U'} 2024-05-09T19:55:35.091 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi/7699763', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 19:49:55.451836', 'locked_by': 'scheduled_suriarte@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOkZpNQCxoniAqwSDaCfTEv1WgrJREDZPuD+6uQW09/0'} 2024-05-09T19:55:35.091 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T19:55:35.097 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-09T19:55:35.097 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-09T19:55:35.097 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T19:55:35.209 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff68d167160>, signals=[15]) 2024-05-09T19:55:35.210 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T19:55:35.216 INFO:teuthology.task.internal:Opening connections... 2024-05-09T19:55:35.216 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T19:55:35.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T19:55:35.299 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-05-09T19:55:35.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T19:55:35.379 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T19:55:35.386 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-05-09T19:55:35.391 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-05-09T19:55:35.392 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-05-09T19:55:35.436 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-09T19:55:35.436 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="Ubuntu" 2024-05-09T19:55:35.436 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="22.04" 2024-05-09T19:55:35.436 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-09T19:55:35.437 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_CODENAME=jammy 2024-05-09T19:55:35.437 INFO:teuthology.orchestra.run.smithi063.stdout:ID=ubuntu 2024-05-09T19:55:35.437 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE=debian 2024-05-09T19:55:35.437 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-09T19:55:35.437 INFO:teuthology.orchestra.run.smithi063.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-09T19:55:35.437 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-09T19:55:35.437 INFO:teuthology.orchestra.run.smithi063.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-09T19:55:35.437 INFO:teuthology.orchestra.run.smithi063.stdout:UBUNTU_CODENAME=jammy 2024-05-09T19:55:35.438 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-05-09T19:55:35.466 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-05-09T19:55:35.471 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-05-09T19:55:35.471 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-05-09T19:55:35.520 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Ubuntu" 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="22.04" 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_CODENAME=jammy 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:ID=ubuntu 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE=debian 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.ubuntu.com/" 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-05-09T19:55:35.521 INFO:teuthology.orchestra.run.smithi120.stdout:UBUNTU_CODENAME=jammy 2024-05-09T19:55:35.522 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-05-09T19:55:35.540 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T19:55:35.550 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T19:55:35.557 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T19:55:35.557 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-05-09T19:55:35.560 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-05-09T19:55:35.569 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T19:55:35.590 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T19:55:35.590 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-05-09T19:55:35.606 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-05-09T19:55:35.618 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T19:55:35.670 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T19:55:35.685 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-05-09T19:55:35.685 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-05-09T19:55:35.685 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T19:55:35.686 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T19:55:35.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T19:55:35.691 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-09T19:55:35.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T19:55:35.692 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-05-09T19:55:35.737 INFO:teuthology.orchestra.run.smithi063.stdout:5.15.0-56-generic 2024-05-09T19:55:35.737 INFO:teuthology.task.kernel:Running kernel on smithi063: 5.15.0-56-generic 2024-05-09T19:55:35.737 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get clean 2024-05-09T19:55:35.738 INFO:teuthology.orchestra.run.smithi120.stdout:5.15.0-56-generic 2024-05-09T19:55:35.738 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.15.0-56-generic 2024-05-09T19:55:35.738 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get clean 2024-05-09T19:55:35.875 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get update 2024-05-09T19:55:35.887 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get update 2024-05-09T19:55:36.016 INFO:teuthology.orchestra.run.smithi120.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-09T19:55:36.027 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-05-09T19:55:36.118 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-09T19:55:36.132 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-09T19:55:36.199 INFO:teuthology.orchestra.run.smithi120.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-05-09T19:55:36.214 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-05-09T19:55:36.246 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [462 kB] 2024-05-09T19:55:36.293 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,427 kB] 2024-05-09T19:55:36.334 INFO:teuthology.orchestra.run.smithi120.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,427 kB] 2024-05-09T19:55:36.389 INFO:teuthology.orchestra.run.smithi120.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [247 kB] 2024-05-09T19:55:36.395 INFO:teuthology.orchestra.run.smithi120.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-05-09T19:55:36.395 INFO:teuthology.orchestra.run.smithi120.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-05-09T19:55:36.397 INFO:teuthology.orchestra.run.smithi120.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,807 kB] 2024-05-09T19:55:36.435 INFO:teuthology.orchestra.run.smithi120.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [307 kB] 2024-05-09T19:55:36.441 INFO:teuthology.orchestra.run.smithi120.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-05-09T19:55:36.442 INFO:teuthology.orchestra.run.smithi120.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [603 kB] 2024-05-09T19:55:36.453 INFO:teuthology.orchestra.run.smithi120.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-05-09T19:55:36.459 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [462 kB] 2024-05-09T19:55:36.471 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [247 kB] 2024-05-09T19:55:36.471 INFO:teuthology.orchestra.run.smithi120.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [164 kB] 2024-05-09T19:55:36.481 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-05-09T19:55:36.482 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-05-09T19:55:36.483 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,807 kB] 2024-05-09T19:55:36.500 INFO:teuthology.orchestra.run.smithi120.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-05-09T19:55:36.501 INFO:teuthology.orchestra.run.smithi120.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-05-09T19:55:36.501 INFO:teuthology.orchestra.run.smithi120.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-05-09T19:55:36.501 INFO:teuthology.orchestra.run.smithi120.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-05-09T19:55:36.535 INFO:teuthology.orchestra.run.smithi120.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-05-09T19:55:36.549 INFO:teuthology.orchestra.run.smithi063.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [307 kB] 2024-05-09T19:55:36.551 INFO:teuthology.orchestra.run.smithi120.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-05-09T19:55:36.562 INFO:teuthology.orchestra.run.smithi063.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-05-09T19:55:36.588 INFO:teuthology.orchestra.run.smithi063.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-05-09T19:55:36.588 INFO:teuthology.orchestra.run.smithi063.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [603 kB] 2024-05-09T19:55:36.593 INFO:teuthology.orchestra.run.smithi063.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-05-09T19:55:36.603 INFO:teuthology.orchestra.run.smithi063.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [164 kB] 2024-05-09T19:55:36.607 INFO:teuthology.orchestra.run.smithi063.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-05-09T19:55:36.607 INFO:teuthology.orchestra.run.smithi063.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-05-09T19:55:36.613 INFO:teuthology.orchestra.run.smithi063.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-05-09T19:55:36.614 INFO:teuthology.orchestra.run.smithi063.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-05-09T19:55:36.629 INFO:teuthology.orchestra.run.smithi063.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-05-09T19:55:36.690 INFO:teuthology.orchestra.run.smithi120.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [624 kB] 2024-05-09T19:55:36.795 INFO:teuthology.orchestra.run.smithi063.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [624 kB] 2024-05-09T19:55:36.889 INFO:teuthology.orchestra.run.smithi120.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,638 kB] 2024-05-09T19:55:37.040 INFO:teuthology.orchestra.run.smithi120.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [307 kB] 2024-05-09T19:55:37.054 INFO:teuthology.orchestra.run.smithi120.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-05-09T19:55:37.055 INFO:teuthology.orchestra.run.smithi120.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,864 kB] 2024-05-09T19:55:37.117 INFO:teuthology.orchestra.run.smithi120.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-05-09T19:55:37.118 INFO:teuthology.orchestra.run.smithi120.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [316 kB] 2024-05-09T19:55:37.125 INFO:teuthology.orchestra.run.smithi120.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-05-09T19:55:37.289 INFO:teuthology.orchestra.run.smithi120.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,074 kB] 2024-05-09T19:55:37.925 INFO:teuthology.orchestra.run.smithi120.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [702 kB] 2024-05-09T19:55:38.385 INFO:teuthology.orchestra.run.smithi063.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,638 kB] 2024-05-09T19:55:38.495 INFO:teuthology.orchestra.run.smithi120.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-05-09T19:55:38.508 INFO:teuthology.orchestra.run.smithi120.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-05-09T19:55:38.509 INFO:teuthology.orchestra.run.smithi120.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-05-09T19:55:38.509 INFO:teuthology.orchestra.run.smithi120.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-05-09T19:55:38.512 INFO:teuthology.orchestra.run.smithi120.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-05-09T19:55:38.512 INFO:teuthology.orchestra.run.smithi120.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-05-09T19:55:38.513 INFO:teuthology.orchestra.run.smithi120.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-05-09T19:55:38.516 INFO:teuthology.orchestra.run.smithi120.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-05-09T19:55:38.519 INFO:teuthology.orchestra.run.smithi120.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-05-09T19:55:38.684 INFO:teuthology.orchestra.run.smithi120.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-05-09T19:55:38.766 INFO:teuthology.orchestra.run.smithi120.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-05-09T19:55:38.882 INFO:teuthology.orchestra.run.smithi120.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-05-09T19:55:38.928 INFO:teuthology.orchestra.run.smithi120.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-05-09T19:55:38.960 INFO:teuthology.orchestra.run.smithi120.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-05-09T19:55:40.204 INFO:teuthology.orchestra.run.smithi063.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [307 kB] 2024-05-09T19:55:40.315 INFO:teuthology.orchestra.run.smithi063.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-05-09T19:55:40.319 INFO:teuthology.orchestra.run.smithi063.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-05-09T19:55:40.333 INFO:teuthology.orchestra.run.smithi063.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,864 kB] 2024-05-09T19:55:42.894 INFO:teuthology.orchestra.run.smithi063.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [316 kB] 2024-05-09T19:55:43.072 INFO:teuthology.orchestra.run.smithi063.stdout:Ign:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en 2024-05-09T19:55:43.237 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 13.4 MB in 3s (4,436 kB/s) 2024-05-09T19:55:43.238 INFO:teuthology.orchestra.run.smithi063.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-05-09T19:55:43.238 INFO:teuthology.orchestra.run.smithi063.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [702 kB] 2024-05-09T19:55:44.674 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-05-09T19:55:44.690 DEBUG:teuthology.orchestra.run.smithi120:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-09T19:55:44.752 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-05-09T19:55:44.953 INFO:teuthology.orchestra.run.smithi120.stdout:Building dependency tree... 2024-05-09T19:55:44.954 INFO:teuthology.orchestra.run.smithi120.stdout:Reading state information... 2024-05-09T19:55:44.987 INFO:teuthology.orchestra.run.smithi063.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,074 kB] 2024-05-09T19:55:45.130 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T19:55:45.130 INFO:teuthology.orchestra.run.smithi120.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T19:55:45.130 INFO:teuthology.orchestra.run.smithi120.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T19:55:45.131 INFO:teuthology.orchestra.run.smithi120.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T19:55:45.131 INFO:teuthology.orchestra.run.smithi120.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-05-09T19:55:45.131 INFO:teuthology.orchestra.run.smithi120.stdout: python2.7-minimal 2024-05-09T19:55:45.131 INFO:teuthology.orchestra.run.smithi120.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T19:55:45.132 INFO:teuthology.orchestra.run.smithi120.stdout:The following additional packages will be installed: 2024-05-09T19:55:45.132 INFO:teuthology.orchestra.run.smithi120.stdout: linux-generic linux-headers-5.15.0-106 linux-headers-5.15.0-106-generic 2024-05-09T19:55:45.132 INFO:teuthology.orchestra.run.smithi120.stdout: linux-headers-generic linux-image-5.15.0-106-generic 2024-05-09T19:55:45.133 INFO:teuthology.orchestra.run.smithi120.stdout: linux-modules-5.15.0-106-generic linux-modules-extra-5.15.0-106-generic 2024-05-09T19:55:45.134 INFO:teuthology.orchestra.run.smithi120.stdout:Suggested packages: 2024-05-09T19:55:45.134 INFO:teuthology.orchestra.run.smithi120.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-05-09T19:55:45.134 INFO:teuthology.orchestra.run.smithi120.stdout:Recommended packages: 2024-05-09T19:55:45.135 INFO:teuthology.orchestra.run.smithi120.stdout: thermald 2024-05-09T19:55:45.185 INFO:teuthology.orchestra.run.smithi120.stdout:The following NEW packages will be installed: 2024-05-09T19:55:45.185 INFO:teuthology.orchestra.run.smithi120.stdout: linux-headers-5.15.0-106 linux-headers-5.15.0-106-generic 2024-05-09T19:55:45.186 INFO:teuthology.orchestra.run.smithi120.stdout: linux-image-5.15.0-106-generic linux-modules-5.15.0-106-generic 2024-05-09T19:55:45.186 INFO:teuthology.orchestra.run.smithi120.stdout: linux-modules-extra-5.15.0-106-generic 2024-05-09T19:55:45.187 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages will be upgraded: 2024-05-09T19:55:45.188 INFO:teuthology.orchestra.run.smithi120.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-09T19:55:45.368 INFO:teuthology.orchestra.run.smithi120.stdout:3 upgraded, 5 newly installed, 0 to remove and 320 not upgraded. 2024-05-09T19:55:45.368 INFO:teuthology.orchestra.run.smithi120.stdout:Need to get 113 MB of archives. 2024-05-09T19:55:45.368 INFO:teuthology.orchestra.run.smithi120.stdout:After this operation, 583 MB of additional disk space will be used. 2024-05-09T19:55:45.368 INFO:teuthology.orchestra.run.smithi120.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-106-generic amd64 5.15.0-106.116 [22.7 MB] 2024-05-09T19:55:47.547 INFO:teuthology.orchestra.run.smithi063.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [245 kB] 2024-05-09T19:55:47.864 INFO:teuthology.orchestra.run.smithi063.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-05-09T19:55:47.963 INFO:teuthology.orchestra.run.smithi063.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-05-09T19:55:47.964 INFO:teuthology.orchestra.run.smithi063.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-05-09T19:55:48.019 INFO:teuthology.orchestra.run.smithi063.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-05-09T19:55:48.028 INFO:teuthology.orchestra.run.smithi063.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-05-09T19:55:48.028 INFO:teuthology.orchestra.run.smithi063.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-05-09T19:55:48.224 INFO:teuthology.orchestra.run.smithi063.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-05-09T19:55:48.545 INFO:teuthology.orchestra.run.smithi063.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-05-09T19:55:48.634 INFO:teuthology.orchestra.run.smithi063.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-05-09T19:55:48.634 INFO:teuthology.orchestra.run.smithi063.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-05-09T19:55:48.720 INFO:teuthology.orchestra.run.smithi063.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-05-09T19:55:48.766 INFO:teuthology.orchestra.run.smithi063.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-05-09T19:55:48.799 INFO:teuthology.orchestra.run.smithi063.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-05-09T19:55:48.800 INFO:teuthology.orchestra.run.smithi063.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [316 kB] 2024-05-09T19:55:53.592 INFO:teuthology.orchestra.run.smithi120.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-106-generic amd64 5.15.0-106.116 [11.5 MB] 2024-05-09T19:55:53.593 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 13.4 MB in 13s (1,017 kB/s) 2024-05-09T19:55:53.988 INFO:teuthology.orchestra.run.smithi120.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-106-generic amd64 5.15.0-106.116 [63.9 MB] 2024-05-09T19:55:55.074 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-09T19:55:55.094 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-09T19:55:55.158 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-09T19:55:55.362 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-05-09T19:55:55.362 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-05-09T19:55:55.544 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T19:55:55.544 INFO:teuthology.orchestra.run.smithi063.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T19:55:55.544 INFO:teuthology.orchestra.run.smithi063.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T19:55:55.545 INFO:teuthology.orchestra.run.smithi063.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T19:55:55.545 INFO:teuthology.orchestra.run.smithi063.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-05-09T19:55:55.545 INFO:teuthology.orchestra.run.smithi063.stdout: python2.7-minimal 2024-05-09T19:55:55.545 INFO:teuthology.orchestra.run.smithi063.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T19:55:55.546 INFO:teuthology.orchestra.run.smithi063.stdout:The following additional packages will be installed: 2024-05-09T19:55:55.546 INFO:teuthology.orchestra.run.smithi063.stdout: linux-generic linux-headers-5.15.0-106 linux-headers-5.15.0-106-generic 2024-05-09T19:55:55.546 INFO:teuthology.orchestra.run.smithi063.stdout: linux-headers-generic linux-image-5.15.0-106-generic 2024-05-09T19:55:55.547 INFO:teuthology.orchestra.run.smithi063.stdout: linux-modules-5.15.0-106-generic linux-modules-extra-5.15.0-106-generic 2024-05-09T19:55:55.548 INFO:teuthology.orchestra.run.smithi063.stdout:Suggested packages: 2024-05-09T19:55:55.548 INFO:teuthology.orchestra.run.smithi063.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-05-09T19:55:55.548 INFO:teuthology.orchestra.run.smithi063.stdout:Recommended packages: 2024-05-09T19:55:55.548 INFO:teuthology.orchestra.run.smithi063.stdout: thermald 2024-05-09T19:55:55.594 INFO:teuthology.orchestra.run.smithi063.stdout:The following NEW packages will be installed: 2024-05-09T19:55:55.594 INFO:teuthology.orchestra.run.smithi063.stdout: linux-headers-5.15.0-106 linux-headers-5.15.0-106-generic 2024-05-09T19:55:55.594 INFO:teuthology.orchestra.run.smithi063.stdout: linux-image-5.15.0-106-generic linux-modules-5.15.0-106-generic 2024-05-09T19:55:55.594 INFO:teuthology.orchestra.run.smithi063.stdout: linux-modules-extra-5.15.0-106-generic 2024-05-09T19:55:55.595 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages will be upgraded: 2024-05-09T19:55:55.596 INFO:teuthology.orchestra.run.smithi063.stdout: linux-generic linux-headers-generic linux-image-generic 2024-05-09T19:55:55.680 INFO:teuthology.orchestra.run.smithi063.stdout:3 upgraded, 5 newly installed, 0 to remove and 320 not upgraded. 2024-05-09T19:55:55.680 INFO:teuthology.orchestra.run.smithi063.stdout:Need to get 113 MB of archives. 2024-05-09T19:55:55.680 INFO:teuthology.orchestra.run.smithi063.stdout:After this operation, 583 MB of additional disk space will be used. 2024-05-09T19:55:55.681 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-106-generic amd64 5.15.0-106.116 [22.7 MB] 2024-05-09T19:55:56.299 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-106-generic amd64 5.15.0-106.116 [11.5 MB] 2024-05-09T19:55:56.558 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-106-generic amd64 5.15.0-106.116 [63.9 MB] 2024-05-09T19:55:57.664 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.106.106 [1,692 B] 2024-05-09T19:55:57.665 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.106.106 [2,502 B] 2024-05-09T19:55:57.665 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-106 all 5.15.0-106.116 [12.3 MB] 2024-05-09T19:55:58.257 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-106-generic amd64 5.15.0-106.116 [2,876 kB] 2024-05-09T19:55:58.353 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.106.106 [2,366 B] 2024-05-09T19:55:58.764 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 113 MB in 3s (41.2 MB/s) 2024-05-09T19:55:58.933 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-modules-5.15.0-106-generic. 2024-05-09T19:56:00.933 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-05-09T19:56:00.940 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../0-linux-modules-5.15.0-106-generic_5.15.0-106.116_amd64.deb ... 2024-05-09T19:56:01.056 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-modules-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:03.458 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-image-5.15.0-106-generic. 2024-05-09T19:56:03.479 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../1-linux-image-5.15.0-106-generic_5.15.0-106.116_amd64.deb ... 2024-05-09T19:56:03.574 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-image-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:04.052 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-106-generic. 2024-05-09T19:56:04.073 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-106-generic_5.15.0-106.116_amd64.deb ... 2024-05-09T19:56:04.110 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-modules-extra-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:05.376 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.106.106 [1,692 B] 2024-05-09T19:56:05.376 INFO:teuthology.orchestra.run.smithi120.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.106.106 [2,502 B] 2024-05-09T19:56:05.377 INFO:teuthology.orchestra.run.smithi120.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-106 all 5.15.0-106.116 [12.3 MB] 2024-05-09T19:56:07.249 INFO:teuthology.orchestra.run.smithi120.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-106-generic amd64 5.15.0-106.116 [2,876 kB] 2024-05-09T19:56:07.300 INFO:teuthology.orchestra.run.smithi120.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.106.106 [2,366 B] 2024-05-09T19:56:07.703 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 113 MB in 22s (5,128 kB/s) 2024-05-09T19:56:07.894 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-modules-5.15.0-106-generic. 2024-05-09T19:56:09.880 INFO:teuthology.orchestra.run.smithi120.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-05-09T19:56:09.884 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../0-linux-modules-5.15.0-106-generic_5.15.0-106.116_amd64.deb ... 2024-05-09T19:56:09.948 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-modules-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:10.827 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../3-linux-generic_5.15.0.106.106_amd64.deb ... 2024-05-09T19:56:10.953 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-generic (5.15.0.106.106) over (5.15.0.56.54) ... 2024-05-09T19:56:11.303 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.106.106_amd64.deb ... 2024-05-09T19:56:11.415 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-image-generic (5.15.0.106.106) over (5.15.0.56.54) ... 2024-05-09T19:56:12.277 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-headers-5.15.0-106. 2024-05-09T19:56:12.299 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../5-linux-headers-5.15.0-106_5.15.0-106.116_all.deb ... 2024-05-09T19:56:12.368 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-headers-5.15.0-106 (5.15.0-106.116) ... 2024-05-09T19:56:13.291 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-image-5.15.0-106-generic. 2024-05-09T19:56:13.312 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../1-linux-image-5.15.0-106-generic_5.15.0-106.116_amd64.deb ... 2024-05-09T19:56:13.401 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-image-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:13.861 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-106-generic. 2024-05-09T19:56:13.881 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-106-generic_5.15.0-106.116_amd64.deb ... 2024-05-09T19:56:13.919 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-modules-extra-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:18.316 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package linux-headers-5.15.0-106-generic. 2024-05-09T19:56:18.353 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../6-linux-headers-5.15.0-106-generic_5.15.0-106.116_amd64.deb ... 2024-05-09T19:56:18.406 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-headers-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:20.878 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../3-linux-generic_5.15.0.106.106_amd64.deb ... 2024-05-09T19:56:20.998 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-generic (5.15.0.106.106) over (5.15.0.56.54) ... 2024-05-09T19:56:21.297 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.106.106_amd64.deb ... 2024-05-09T19:56:21.313 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.106.106_amd64.deb ... 2024-05-09T19:56:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking linux-headers-generic (5.15.0.106.106) over (5.15.0.56.54) ... 2024-05-09T19:56:21.435 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-image-generic (5.15.0.106.106) over (5.15.0.56.54) ... 2024-05-09T19:56:21.727 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-headers-5.15.0-106 (5.15.0-106.116) ... 2024-05-09T19:56:21.738 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-headers-5.15.0-106. 2024-05-09T19:56:21.760 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../5-linux-headers-5.15.0-106_5.15.0-106.116_all.deb ... 2024-05-09T19:56:21.805 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-headers-5.15.0-106 (5.15.0-106.116) ... 2024-05-09T19:56:21.856 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-headers-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:21.982 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-headers-generic (5.15.0.106.106) ... 2024-05-09T19:56:22.110 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-image-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:23.458 INFO:teuthology.orchestra.run.smithi063.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-05-09T19:56:23.459 INFO:teuthology.orchestra.run.smithi063.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-05-09T19:56:23.459 INFO:teuthology.orchestra.run.smithi063.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-106-generic 2024-05-09T19:56:23.459 INFO:teuthology.orchestra.run.smithi063.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-106-generic 2024-05-09T19:56:23.637 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-modules-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:25.063 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-modules-extra-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:26.506 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-image-generic (5.15.0.106.106) ... 2024-05-09T19:56:26.632 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up linux-generic (5.15.0.106.106) ... 2024-05-09T19:56:26.792 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for linux-image-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:26.853 INFO:teuthology.orchestra.run.smithi063.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-09T19:56:26.854 INFO:teuthology.orchestra.run.smithi063.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-106-generic 2024-05-09T19:56:27.573 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-headers-5.15.0-106-generic. 2024-05-09T19:56:27.610 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../6-linux-headers-5.15.0-106-generic_5.15.0-106.116_amd64.deb ... 2024-05-09T19:56:27.648 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-headers-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:30.264 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.106.106_amd64.deb ... 2024-05-09T19:56:30.390 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-headers-generic (5.15.0.106.106) over (5.15.0.56.54) ... 2024-05-09T19:56:30.724 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-headers-5.15.0-106 (5.15.0-106.116) ... 2024-05-09T19:56:30.845 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-headers-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:30.972 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-headers-generic (5.15.0.106.106) ... 2024-05-09T19:56:31.092 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-image-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:32.393 INFO:teuthology.orchestra.run.smithi120.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-05-09T19:56:32.393 INFO:teuthology.orchestra.run.smithi120.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-05-09T19:56:32.393 INFO:teuthology.orchestra.run.smithi120.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-106-generic 2024-05-09T19:56:32.393 INFO:teuthology.orchestra.run.smithi120.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-106-generic 2024-05-09T19:56:32.561 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-modules-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:33.980 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-modules-extra-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:35.373 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-image-generic (5.15.0.106.106) ... 2024-05-09T19:56:35.482 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-generic (5.15.0.106.106) ... 2024-05-09T19:56:35.584 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for linux-image-5.15.0-106-generic (5.15.0-106.116) ... 2024-05-09T19:56:35.639 INFO:teuthology.orchestra.run.smithi120.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-05-09T19:56:35.639 INFO:teuthology.orchestra.run.smithi120.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-106-generic 2024-05-09T19:56:41.369 INFO:teuthology.orchestra.run.smithi063.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-09T19:56:41.369 INFO:teuthology.orchestra.run.smithi063.stdout:Sourcing file `/etc/default/grub' 2024-05-09T19:56:41.391 INFO:teuthology.orchestra.run.smithi063.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-09T19:56:41.406 INFO:teuthology.orchestra.run.smithi063.stdout:Generating grub configuration file ... 2024-05-09T19:56:41.754 INFO:teuthology.orchestra.run.smithi063.stdout:Found linux image: /boot/vmlinuz-5.15.0-106-generic 2024-05-09T19:56:41.770 INFO:teuthology.orchestra.run.smithi063.stdout:Found initrd image: /boot/initrd.img-5.15.0-106-generic 2024-05-09T19:56:42.191 INFO:teuthology.orchestra.run.smithi063.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-09T19:56:42.197 INFO:teuthology.orchestra.run.smithi063.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-09T19:56:42.269 INFO:teuthology.orchestra.run.smithi063.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-09T19:56:42.274 INFO:teuthology.orchestra.run.smithi063.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-09T19:56:42.423 INFO:teuthology.orchestra.run.smithi063.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-09T19:56:42.423 INFO:teuthology.orchestra.run.smithi063.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-05-09T19:56:42.423 INFO:teuthology.orchestra.run.smithi063.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-09T19:56:42.461 INFO:teuthology.orchestra.run.smithi063.stdout:done 2024-05-09T19:56:42.971 DEBUG:teuthology.orchestra.run.smithi063:> dpkg -s linux-image-generic 2024-05-09T19:56:42.997 INFO:teuthology.orchestra.run.smithi063.stdout:Package: linux-image-generic 2024-05-09T19:56:42.997 INFO:teuthology.orchestra.run.smithi063.stdout:Status: install ok installed 2024-05-09T19:56:42.997 INFO:teuthology.orchestra.run.smithi063.stdout:Priority: optional 2024-05-09T19:56:42.997 INFO:teuthology.orchestra.run.smithi063.stdout:Section: kernel 2024-05-09T19:56:42.997 INFO:teuthology.orchestra.run.smithi063.stdout:Installed-Size: 21 2024-05-09T19:56:42.998 INFO:teuthology.orchestra.run.smithi063.stdout:Maintainer: Ubuntu Kernel Team 2024-05-09T19:56:42.998 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: amd64 2024-05-09T19:56:42.998 INFO:teuthology.orchestra.run.smithi063.stdout:Source: linux-meta 2024-05-09T19:56:42.998 INFO:teuthology.orchestra.run.smithi063.stdout:Version: 5.15.0.106.106 2024-05-09T19:56:42.998 INFO:teuthology.orchestra.run.smithi063.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-106), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-09T19:56:42.998 INFO:teuthology.orchestra.run.smithi063.stdout:Depends: linux-image-5.15.0-106-generic, linux-modules-extra-5.15.0-106-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-09T19:56:42.998 INFO:teuthology.orchestra.run.smithi063.stdout:Recommends: thermald 2024-05-09T19:56:42.998 INFO:teuthology.orchestra.run.smithi063.stdout:Description: Generic Linux kernel image 2024-05-09T19:56:42.999 INFO:teuthology.orchestra.run.smithi063.stdout: This package will always depend on the latest generic kernel image 2024-05-09T19:56:42.999 INFO:teuthology.orchestra.run.smithi063.stdout: available. 2024-05-09T19:56:42.999 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-106-generic 2024-05-09T19:56:42.999 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T19:56:42.999 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T19:56:42.999 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-09T19:56:42.999 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi063.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T19:56:42.999 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get clean 2024-05-09T19:56:43.156 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get update 2024-05-09T19:56:43.397 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-09T19:56:43.482 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-09T19:56:43.482 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-09T19:56:43.554 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-09T19:56:44.826 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-09T19:56:44.845 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-09T19:56:44.909 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-09T19:56:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-05-09T19:56:45.112 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-05-09T19:56:45.295 INFO:teuthology.orchestra.run.smithi063.stdout:linux-image-generic is already the newest version (5.15.0.106.106). 2024-05-09T19:56:45.295 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T19:56:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T19:56:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T19:56:45.296 INFO:teuthology.orchestra.run.smithi063.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T19:56:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-09T19:56:45.297 INFO:teuthology.orchestra.run.smithi063.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-09T19:56:45.297 INFO:teuthology.orchestra.run.smithi063.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T19:56:45.340 INFO:teuthology.orchestra.run.smithi063.stdout:0 upgraded, 0 newly installed, 0 to remove and 320 not upgraded. 2024-05-09T19:56:45.342 DEBUG:teuthology.orchestra.run.smithi063:> dpkg -s linux-image-generic 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Package: linux-image-generic 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Status: install ok installed 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Priority: optional 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Section: kernel 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Installed-Size: 21 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Maintainer: Ubuntu Kernel Team 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: amd64 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Source: linux-meta 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Version: 5.15.0.106.106 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-106), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Depends: linux-image-5.15.0-106-generic, linux-modules-extra-5.15.0-106-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-09T19:56:45.358 INFO:teuthology.orchestra.run.smithi063.stdout:Recommends: thermald 2024-05-09T19:56:45.359 INFO:teuthology.orchestra.run.smithi063.stdout:Description: Generic Linux kernel image 2024-05-09T19:56:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: This package will always depend on the latest generic kernel image 2024-05-09T19:56:45.359 INFO:teuthology.orchestra.run.smithi063.stdout: available. 2024-05-09T19:56:45.360 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-05-09T19:56:45.404 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.hKXUVQMI91 2024-05-09T19:56:45.405 DEBUG:teuthology.orchestra.run.smithi063:> sudo cp /boot/grub/grub.cfg /tmp/tmp.hKXUVQMI91 2024-05-09T19:56:45.480 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.hKXUVQMI91 2024-05-09T19:56:45.604 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.hKXUVQMI91 is 10KB 2024-05-09T19:56:45.618 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.hKXUVQMI91 2024-05-09T19:56:45.624 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-09T19:56:45.676 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T19:56:45.677 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-09T19:56:45.677 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-09T19:56:45.756 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-106-generic 2024-05-09T19:56:45.756 DEBUG:teuthology.orchestra.run.smithi063:> sudo update-grub 2024-05-09T19:56:47.063 INFO:teuthology.orchestra.run.smithi063.stderr:Sourcing file `/etc/default/grub' 2024-05-09T19:56:47.064 INFO:teuthology.orchestra.run.smithi063.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-09T19:56:47.067 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-05-09T19:56:47.256 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.15.0-106-generic 2024-05-09T19:56:47.263 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.15.0-106-generic 2024-05-09T19:56:47.620 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-09T19:56:47.626 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-09T19:56:47.669 INFO:teuthology.orchestra.run.smithi063.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-09T19:56:47.675 INFO:teuthology.orchestra.run.smithi063.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-09T19:56:47.820 INFO:teuthology.orchestra.run.smithi063.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-09T19:56:47.820 INFO:teuthology.orchestra.run.smithi063.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-09T19:56:47.820 INFO:teuthology.orchestra.run.smithi063.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-09T19:56:47.835 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-05-09T19:56:47.860 DEBUG:teuthology.orchestra.run.smithi063:> sudo shutdown -r now 2024-05-09T19:56:49.494 INFO:teuthology.orchestra.run.smithi120.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-05-09T19:56:49.494 INFO:teuthology.orchestra.run.smithi120.stdout:Sourcing file `/etc/default/grub' 2024-05-09T19:56:49.509 INFO:teuthology.orchestra.run.smithi120.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-09T19:56:49.531 INFO:teuthology.orchestra.run.smithi120.stdout:Generating grub configuration file ... 2024-05-09T19:56:49.780 INFO:teuthology.orchestra.run.smithi120.stdout:Found linux image: /boot/vmlinuz-5.15.0-106-generic 2024-05-09T19:56:49.793 INFO:teuthology.orchestra.run.smithi120.stdout:Found initrd image: /boot/initrd.img-5.15.0-106-generic 2024-05-09T19:56:50.084 INFO:teuthology.orchestra.run.smithi120.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-09T19:56:50.089 INFO:teuthology.orchestra.run.smithi120.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-09T19:56:50.153 INFO:teuthology.orchestra.run.smithi120.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-09T19:56:50.159 INFO:teuthology.orchestra.run.smithi120.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-09T19:56:50.334 INFO:teuthology.orchestra.run.smithi120.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-09T19:56:50.334 INFO:teuthology.orchestra.run.smithi120.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-05-09T19:56:50.334 INFO:teuthology.orchestra.run.smithi120.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-09T19:56:50.382 INFO:teuthology.orchestra.run.smithi120.stdout:done 2024-05-09T19:56:50.893 DEBUG:teuthology.orchestra.run.smithi120:> dpkg -s linux-image-generic 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Package: linux-image-generic 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Status: install ok installed 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Priority: optional 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Section: kernel 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Installed-Size: 21 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Maintainer: Ubuntu Kernel Team 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: amd64 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Source: linux-meta 2024-05-09T19:56:50.916 INFO:teuthology.orchestra.run.smithi120.stdout:Version: 5.15.0.106.106 2024-05-09T19:56:50.917 INFO:teuthology.orchestra.run.smithi120.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-106), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-09T19:56:50.917 INFO:teuthology.orchestra.run.smithi120.stdout:Depends: linux-image-5.15.0-106-generic, linux-modules-extra-5.15.0-106-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-09T19:56:50.917 INFO:teuthology.orchestra.run.smithi120.stdout:Recommends: thermald 2024-05-09T19:56:50.917 INFO:teuthology.orchestra.run.smithi120.stdout:Description: Generic Linux kernel image 2024-05-09T19:56:50.917 INFO:teuthology.orchestra.run.smithi120.stdout: This package will always depend on the latest generic kernel image 2024-05-09T19:56:50.917 INFO:teuthology.orchestra.run.smithi120.stdout: available. 2024-05-09T19:56:50.917 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-106-generic 2024-05-09T19:56:50.917 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T19:56:50.918 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T19:56:50.918 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-09T19:56:50.918 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T19:56:50.918 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get clean 2024-05-09T19:56:51.087 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get update 2024-05-09T19:56:51.210 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-09T19:56:51.218 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-09T19:56:51.236 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-09T19:56:51.321 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-09T19:56:52.603 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-05-09T19:56:52.622 DEBUG:teuthology.orchestra.run.smithi120:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-05-09T19:56:52.682 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-05-09T19:56:52.895 INFO:teuthology.orchestra.run.smithi120.stdout:Building dependency tree... 2024-05-09T19:56:52.895 INFO:teuthology.orchestra.run.smithi120.stdout:Reading state information... 2024-05-09T19:56:53.075 INFO:teuthology.orchestra.run.smithi120.stdout:linux-image-generic is already the newest version (5.15.0.106.106). 2024-05-09T19:56:53.075 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T19:56:53.075 INFO:teuthology.orchestra.run.smithi120.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T19:56:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T19:56:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T19:56:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-09T19:56:53.076 INFO:teuthology.orchestra.run.smithi120.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-09T19:56:53.076 INFO:teuthology.orchestra.run.smithi120.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T19:56:53.116 INFO:teuthology.orchestra.run.smithi120.stdout:0 upgraded, 0 newly installed, 0 to remove and 320 not upgraded. 2024-05-09T19:56:53.118 DEBUG:teuthology.orchestra.run.smithi120:> dpkg -s linux-image-generic 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Package: linux-image-generic 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Status: install ok installed 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Priority: optional 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Section: kernel 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Installed-Size: 21 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Maintainer: Ubuntu Kernel Team 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: amd64 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Source: linux-meta 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Version: 5.15.0.106.106 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-106), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Depends: linux-image-5.15.0-106-generic, linux-modules-extra-5.15.0-106-generic, linux-firmware, intel-microcode, amd64-microcode 2024-05-09T19:56:53.135 INFO:teuthology.orchestra.run.smithi120.stdout:Recommends: thermald 2024-05-09T19:56:53.136 INFO:teuthology.orchestra.run.smithi120.stdout:Description: Generic Linux kernel image 2024-05-09T19:56:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: This package will always depend on the latest generic kernel image 2024-05-09T19:56:53.136 INFO:teuthology.orchestra.run.smithi120.stdout: available. 2024-05-09T19:56:53.136 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-05-09T19:56:53.180 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.MPuSIOkOHV 2024-05-09T19:56:53.180 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub/grub.cfg /tmp/tmp.MPuSIOkOHV 2024-05-09T19:56:53.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.MPuSIOkOHV 2024-05-09T19:56:53.375 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.MPuSIOkOHV is 10KB 2024-05-09T19:56:53.399 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.MPuSIOkOHV 2024-05-09T19:56:53.406 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-05-09T19:56:53.463 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T19:56:53.463 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-05-09T19:56:53.463 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-05-09T19:56:53.563 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-106-generic 2024-05-09T19:56:53.563 DEBUG:teuthology.orchestra.run.smithi120:> sudo update-grub 2024-05-09T19:56:54.796 INFO:teuthology.orchestra.run.smithi120.stderr:Sourcing file `/etc/default/grub' 2024-05-09T19:56:54.797 INFO:teuthology.orchestra.run.smithi120.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-05-09T19:56:54.799 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2024-05-09T19:56:54.990 INFO:teuthology.orchestra.run.smithi120.stderr:Found linux image: /boot/vmlinuz-5.15.0-106-generic 2024-05-09T19:56:54.995 INFO:teuthology.orchestra.run.smithi120.stderr:Found initrd image: /boot/initrd.img-5.15.0-106-generic 2024-05-09T19:56:55.230 INFO:teuthology.orchestra.run.smithi120.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-05-09T19:56:55.235 INFO:teuthology.orchestra.run.smithi120.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-05-09T19:56:55.279 INFO:teuthology.orchestra.run.smithi120.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-05-09T19:56:55.284 INFO:teuthology.orchestra.run.smithi120.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-05-09T19:56:55.410 INFO:teuthology.orchestra.run.smithi120.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-05-09T19:56:55.410 INFO:teuthology.orchestra.run.smithi120.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-05-09T19:56:55.410 INFO:teuthology.orchestra.run.smithi120.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-05-09T19:56:55.431 INFO:teuthology.orchestra.run.smithi120.stderr:done 2024-05-09T19:56:55.449 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2024-05-09T19:57:17.886 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-09T19:57:17.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T19:57:17.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T19:57:25.458 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-09T19:57:25.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-05-09T19:57:25.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T19:57:52.543 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-09T19:58:01.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T19:58:01.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T19:58:19.934 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-09T19:58:25.466 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T19:58:31.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T19:58:31.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T19:58:31.945 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-05-09T19:58:34.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-05-09T19:58:34.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T19:58:35.613 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-05-09T19:58:36.449 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-05-09T19:58:36.449 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-106-generic"... 2024-05-09T19:58:36.449 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-05-09T19:58:36.497 INFO:teuthology.orchestra.run.smithi120.stdout:5.15.0-106-generic 2024-05-09T19:58:36.497 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-106-generic vs 5.15.0-106-generic 2024-05-09T19:58:36.497 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T19:58:36.497 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-09T19:58:37.497 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-09T19:58:37.498 DEBUG:teuthology.orchestra.run.smithi120:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T19:58:37.672 INFO:teuthology.orchestra.run.smithi120.stdout:ttyS1 2024-05-09T19:58:37.718 DEBUG:teuthology.parallel:result is None 2024-05-09T19:58:46.946 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T19:58:46.947 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T19:58:47.148 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-09T19:58:47.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T19:58:47.709 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-106-generic"... 2024-05-09T19:58:47.709 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-05-09T19:58:47.757 INFO:teuthology.orchestra.run.smithi063.stdout:5.15.0-106-generic 2024-05-09T19:58:47.757 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-106-generic vs 5.15.0-106-generic 2024-05-09T19:58:47.757 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T19:58:47.757 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-05-09T19:58:48.758 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-09T19:58:48.758 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T19:58:48.832 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-05-09T19:58:48.867 DEBUG:teuthology.parallel:result is None 2024-05-09T19:58:48.868 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T19:58:48.876 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T19:58:48.876 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T19:58:48.879 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T19:58:48.885 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T19:58:48.927 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T19:58:48.957 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T19:58:48.957 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T19:58:48.960 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T19:58:49.048 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T19:58:49.058 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T19:58:49.058 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T19:58:49.061 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T19:58:49.081 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T19:58:49.082 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T19:58:49.089 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T19:58:49.090 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T19:58:49.091 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T19:58:49.099 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T19:58:49.099 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T19:58:49.134 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T19:58:49.148 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T19:58:49.157 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T19:58:49.158 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T19:58:49.190 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T19:58:49.197 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T19:58:49.241 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T19:58:49.289 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T19:58:49.289 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T19:58:49.350 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T19:58:49.356 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T19:58:49.404 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T19:58:49.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T19:58:49.472 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-05-09T19:58:49.475 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-05-09T19:58:49.544 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T19:58:49.552 INFO:teuthology.task.internal:Starting timer... 2024-05-09T19:58:49.552 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T19:58:49.647 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T19:58:49.667 DEBUG:teuthology.task.selinux:Excluding smithi063: OS 'ubuntu' does not support SELinux 2024-05-09T19:58:49.667 DEBUG:teuthology.task.selinux:Excluding smithi120: OS 'ubuntu' does not support SELinux 2024-05-09T19:58:49.668 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T19:58:49.668 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-05-09T19:58:49.668 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T19:58:49.668 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T19:58:49.723 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T19:58:49.741 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T19:58:49.743 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi063.front.sepia.ceph.com,smithi120.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T20:05:58.587 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi063.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com')] 2024-05-09T20:05:58.588 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T20:05:58.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T20:05:58.673 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-09T20:05:58.750 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T20:05:58.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-05-09T20:05:58.751 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T20:05:58.831 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-05-09T20:05:58.910 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-05-09T20:05:58.910 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T20:05:58.920 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T20:05:58.920 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T20:05:58.921 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T20:05:58.923 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T20:05:58.923 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T20:05:58.951 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-09T20:05:58.951 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Command line: ntpd -gq 2024-05-09T20:05:58.951 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: ---------------------------------------------------- 2024-05-09T20:05:58.951 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: ntp-4 is maintained by Network Time Foundation, 2024-05-09T20:05:58.951 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-09T20:05:58.952 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: corporation. Support and training for ntp-4 are 2024-05-09T20:05:58.952 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: available at https://www.nwtime.org/support 2024-05-09T20:05:58.952 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: ---------------------------------------------------- 2024-05-09T20:05:58.952 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: proto: precision = 0.065 usec (-24) 2024-05-09T20:05:58.952 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: basedate set to 2022-02-04 2024-05-09T20:05:58.953 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: gps base set to 2022-02-06 (week 2196) 2024-05-09T20:05:58.953 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-09T20:05:58.953 INFO:teuthology.orchestra.run.smithi063.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-09T20:05:58.953 INFO:teuthology.orchestra.run.smithi063.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-09T20:05:58.954 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: restrict ::: KOD does nothing without LIMITED. 2024-05-09T20:05:58.954 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Listen and drop on 0 v6wildcard [::]:123 2024-05-09T20:05:58.954 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-09T20:05:58.954 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Listen normally on 2 lo 127.0.0.1:123 2024-05-09T20:05:58.954 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Listen normally on 3 enp3s0f1 172.21.15.63:123 2024-05-09T20:05:58.954 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Listen normally on 4 lo [::1]:123 2024-05-09T20:05:58.954 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:feb6:f8d7%5]:123 2024-05-09T20:05:58.955 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:05:58 ntpd[17245]: Listening on routing socket on fd #22 for interface updates 2024-05-09T20:05:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-05-09T20:05:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Command line: ntpd -gq 2024-05-09T20:05:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: ---------------------------------------------------- 2024-05-09T20:05:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: ntp-4 is maintained by Network Time Foundation, 2024-05-09T20:05:58.980 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-05-09T20:05:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: corporation. Support and training for ntp-4 are 2024-05-09T20:05:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: available at https://www.nwtime.org/support 2024-05-09T20:05:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: ---------------------------------------------------- 2024-05-09T20:05:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: proto: precision = 0.046 usec (-24) 2024-05-09T20:05:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: basedate set to 2022-02-04 2024-05-09T20:05:58.981 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: gps base set to 2022-02-06 (week 2196) 2024-05-09T20:05:58.982 INFO:teuthology.orchestra.run.smithi120.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-09T20:05:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-05-09T20:05:58.982 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: restrict ::: KOD does nothing without LIMITED. 2024-05-09T20:05:58.982 INFO:teuthology.orchestra.run.smithi120.stderr:restrict ::: KOD does nothing without LIMITED. 2024-05-09T20:05:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Listen and drop on 0 v6wildcard [::]:123 2024-05-09T20:05:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-05-09T20:05:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Listen normally on 2 lo 127.0.0.1:123 2024-05-09T20:05:58.983 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Listen normally on 3 enp3s0f1 172.21.15.120:123 2024-05-09T20:05:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Listen normally on 4 lo [::1]:123 2024-05-09T20:05:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:478f%5]:123 2024-05-09T20:05:58.984 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:05:58 ntpd[17273]: Listening on routing socket on fd #22 for interface updates 2024-05-09T20:05:59.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:05:59 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:05:59.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:05:59 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:05:59.981 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:05:59.981 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:05:59.981 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:05:59.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:05:59 ntpd[17273]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:01.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:01 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:01.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:01 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:01.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:01 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:01.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:01 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:01.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:01 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:01.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:01 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:01.981 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:01.981 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:01.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:01 ntpd[17273]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:02.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:02 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:02.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:02 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:02.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:02 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:02.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:02 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:02.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:02 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:02.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:02 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:03.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:03.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:03 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:03.984 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.984 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.984 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.984 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.984 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.984 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:03.985 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:03 ntpd[17273]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:04.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:04 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:04.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:04 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:04.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:04 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:04.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:04 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:04.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:04 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:04.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:04 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:05.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:05.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:05.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:05 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:05.981 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:05.981 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:05.981 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stdout: 9 May 20:06:05 ntpd[17273]: ntpd: time slew -0.000156 s 2024-05-09T20:06:05.982 INFO:teuthology.orchestra.run.smithi120.stdout:ntpd: time slew -0.000156s 2024-05-09T20:06:05.983 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-09T20:06:05.983 INFO:teuthology.orchestra.run.smithi120.stderr: 9 May 20:06:05 ntpd[17273]: can't open /var/log/ntpstats/loopstats.20240509: Permission denied 2024-05-09T20:06:06.122 INFO:teuthology.orchestra.run.smithi120.stdout: remote refid st t when poll reach delay offset jitter 2024-05-09T20:06:06.122 INFO:teuthology.orchestra.run.smithi120.stdout:============================================================================== 2024-05-09T20:06:06.122 INFO:teuthology.orchestra.run.smithi120.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-09T20:06:06.122 INFO:teuthology.orchestra.run.smithi120.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-09T20:06:06.122 INFO:teuthology.orchestra.run.smithi120.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-09T20:06:06.122 INFO:teuthology.orchestra.run.smithi120.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-09T20:06:06.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:06 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:06.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:06 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:06.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:06 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:06.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:06 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:06.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:06 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:06.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:06 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:07.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:07.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:07.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:07.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:07.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-05-09T20:06:07.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: can't open /var/log/ntpstats/rawstats.20240509: Permission denied 2024-05-09T20:06:07.952 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-05-09T20:06:07.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: can't open /var/log/ntpstats/peerstats.20240509: Permission denied 2024-05-09T20:06:07.953 INFO:teuthology.orchestra.run.smithi063.stdout: 9 May 20:06:07 ntpd[17245]: ntpd: time slew +0.001187 s 2024-05-09T20:06:07.953 INFO:teuthology.orchestra.run.smithi063.stdout:ntpd: time slew +0.001187s 2024-05-09T20:06:07.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-05-09T20:06:07.953 INFO:teuthology.orchestra.run.smithi063.stderr: 9 May 20:06:07 ntpd[17245]: can't open /var/log/ntpstats/loopstats.20240509: Permission denied 2024-05-09T20:06:08.001 INFO:teuthology.orchestra.run.smithi063.stdout: remote refid st t when poll reach delay offset jitter 2024-05-09T20:06:08.001 INFO:teuthology.orchestra.run.smithi063.stdout:============================================================================== 2024-05-09T20:06:08.002 INFO:teuthology.orchestra.run.smithi063.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-09T20:06:08.002 INFO:teuthology.orchestra.run.smithi063.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-09T20:06:08.002 INFO:teuthology.orchestra.run.smithi063.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-09T20:06:08.002 INFO:teuthology.orchestra.run.smithi063.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-05-09T20:06:08.002 INFO:teuthology.run_tasks:Running task install... 2024-05-09T20:06:08.011 DEBUG:teuthology.task.install:project ceph 2024-05-09T20:06:08.011 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '04969094632d400ff0930fc029b1d19b9c187eef'}} 2024-05-09T20:06:08.011 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '04969094632d400ff0930fc029b1d19b9c187eef'} 2024-05-09T20:06:08.011 INFO:teuthology.task.install:Using flavor: default 2024-05-09T20:06:08.017 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T20:06:08.017 INFO:teuthology.task.install:extra packages: [] 2024-05-09T20:06:08.017 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-key list | grep Ceph 2024-05-09T20:06:08.018 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-key list | grep Ceph 2024-05-09T20:06:08.081 INFO:teuthology.orchestra.run.smithi120.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-09T20:06:08.098 INFO:teuthology.orchestra.run.smithi063.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-05-09T20:06:08.110 INFO:teuthology.orchestra.run.smithi120.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-09T20:06:08.110 INFO:teuthology.orchestra.run.smithi120.stdout:uid [ unknown] Ceph.com (release key) 2024-05-09T20:06:08.111 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-05-09T20:06:08.111 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-05-09T20:06:08.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=04969094632d400ff0930fc029b1d19b9c187eef 2024-05-09T20:06:08.127 INFO:teuthology.orchestra.run.smithi063.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-05-09T20:06:08.127 INFO:teuthology.orchestra.run.smithi063.stdout:uid [ unknown] Ceph.com (release key) 2024-05-09T20:06:08.127 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-05-09T20:06:08.127 INFO:teuthology.task.install.deb:Installing system (non-project) packages: s3cmd, maven on remote deb x86_64 2024-05-09T20:06:08.127 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=04969094632d400ff0930fc029b1d19b9c187eef 2024-05-09T20:06:08.266 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default/ 2024-05-09T20:06:08.266 INFO:teuthology.task.install.deb:Package version is 19.0.0-3307-g04969094-1jammy 2024-05-09T20:06:08.433 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default/ 2024-05-09T20:06:08.433 INFO:teuthology.task.install.deb:Package version is 19.0.0-3307-g04969094-1jammy 2024-05-09T20:06:08.535 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:06:08.535 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-09T20:06:08.543 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:06:08.543 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-05-09T20:06:08.550 DEBUG:teuthology.orchestra.run.smithi063:> sudo apt-get update 2024-05-09T20:06:08.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get update 2024-05-09T20:06:08.776 INFO:teuthology.orchestra.run.smithi120.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy InRelease 2024-05-09T20:06:08.787 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-09T20:06:08.797 INFO:teuthology.orchestra.run.smithi120.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-05-09T20:06:08.820 INFO:teuthology.orchestra.run.smithi120.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-09T20:06:08.841 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-05-09T20:06:08.860 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-05-09T20:06:08.863 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-09T20:06:08.881 INFO:teuthology.orchestra.run.smithi120.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,401 B] 2024-05-09T20:06:08.912 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-09T20:06:08.925 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-05-09T20:06:08.956 INFO:teuthology.orchestra.run.smithi063.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy InRelease 2024-05-09T20:06:08.962 INFO:teuthology.orchestra.run.smithi063.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-09T20:06:09.007 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-05-09T20:06:09.021 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy Release [6,978 B] 2024-05-09T20:06:09.088 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-05-09T20:06:09.089 INFO:teuthology.orchestra.run.smithi063.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy Release.gpg 2024-05-09T20:06:09.162 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-05-09T20:06:09.241 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 28.5 kB in 1s (56.6 kB/s) 2024-05-09T20:06:09.244 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,401 B] 2024-05-09T20:06:09.342 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 28.5 kB in 1s (48.7 kB/s) 2024-05-09T20:06:10.257 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-05-09T20:06:10.274 DEBUG:teuthology.orchestra.run.smithi120:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3307-g04969094-1jammy cephadm=19.0.0-3307-g04969094-1jammy ceph-mds=19.0.0-3307-g04969094-1jammy ceph-mgr=19.0.0-3307-g04969094-1jammy ceph-common=19.0.0-3307-g04969094-1jammy ceph-fuse=19.0.0-3307-g04969094-1jammy ceph-test=19.0.0-3307-g04969094-1jammy ceph-volume=19.0.0-3307-g04969094-1jammy radosgw=19.0.0-3307-g04969094-1jammy python3-rados=19.0.0-3307-g04969094-1jammy python3-rgw=19.0.0-3307-g04969094-1jammy python3-cephfs=19.0.0-3307-g04969094-1jammy python3-rbd=19.0.0-3307-g04969094-1jammy libcephfs2=19.0.0-3307-g04969094-1jammy libcephfs-dev=19.0.0-3307-g04969094-1jammy librados2=19.0.0-3307-g04969094-1jammy librbd1=19.0.0-3307-g04969094-1jammy rbd-fuse=19.0.0-3307-g04969094-1jammy 2024-05-09T20:06:10.325 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-05-09T20:06:10.349 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-09T20:06:10.365 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3307-g04969094-1jammy cephadm=19.0.0-3307-g04969094-1jammy ceph-mds=19.0.0-3307-g04969094-1jammy ceph-mgr=19.0.0-3307-g04969094-1jammy ceph-common=19.0.0-3307-g04969094-1jammy ceph-fuse=19.0.0-3307-g04969094-1jammy ceph-test=19.0.0-3307-g04969094-1jammy ceph-volume=19.0.0-3307-g04969094-1jammy radosgw=19.0.0-3307-g04969094-1jammy python3-rados=19.0.0-3307-g04969094-1jammy python3-rgw=19.0.0-3307-g04969094-1jammy python3-cephfs=19.0.0-3307-g04969094-1jammy python3-rbd=19.0.0-3307-g04969094-1jammy libcephfs2=19.0.0-3307-g04969094-1jammy libcephfs-dev=19.0.0-3307-g04969094-1jammy librados2=19.0.0-3307-g04969094-1jammy librbd1=19.0.0-3307-g04969094-1jammy rbd-fuse=19.0.0-3307-g04969094-1jammy 2024-05-09T20:06:10.426 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-09T20:06:10.503 INFO:teuthology.orchestra.run.smithi120.stdout:Building dependency tree... 2024-05-09T20:06:10.503 INFO:teuthology.orchestra.run.smithi120.stdout:Reading state information... 2024-05-09T20:06:10.630 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-05-09T20:06:10.631 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-05-09T20:06:10.674 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T20:06:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T20:06:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T20:06:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T20:06:10.674 INFO:teuthology.orchestra.run.smithi120.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-09T20:06:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-09T20:06:10.675 INFO:teuthology.orchestra.run.smithi120.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T20:06:10.675 INFO:teuthology.orchestra.run.smithi120.stdout:The following additional packages will be installed: 2024-05-09T20:06:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-09T20:06:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-09T20:06:10.675 INFO:teuthology.orchestra.run.smithi120.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-09T20:06:10.676 INFO:teuthology.orchestra.run.smithi120.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-09T20:06:10.677 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet zip 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout:Suggested packages: 2024-05-09T20:06:10.678 INFO:teuthology.orchestra.run.smithi120.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-09T20:06:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-09T20:06:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-09T20:06:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-09T20:06:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-09T20:06:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-09T20:06:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-09T20:06:10.679 INFO:teuthology.orchestra.run.smithi120.stdout:Recommended packages: 2024-05-09T20:06:10.679 INFO:teuthology.orchestra.run.smithi120.stdout: btrfs-tools 2024-05-09T20:06:10.779 INFO:teuthology.orchestra.run.smithi120.stdout:The following NEW packages will be installed: 2024-05-09T20:06:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-09T20:06:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-09T20:06:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-05-09T20:06:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-05-09T20:06:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-05-09T20:06:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-05-09T20:06:10.779 INFO:teuthology.orchestra.run.smithi120.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-05-09T20:06:10.780 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-05-09T20:06:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-05-09T20:06:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-05-09T20:06:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-05-09T20:06:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-05-09T20:06:10.781 INFO:teuthology.orchestra.run.smithi120.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-05-09T20:06:10.832 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T20:06:10.832 INFO:teuthology.orchestra.run.smithi063.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T20:06:10.833 INFO:teuthology.orchestra.run.smithi063.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T20:06:10.833 INFO:teuthology.orchestra.run.smithi063.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T20:06:10.833 INFO:teuthology.orchestra.run.smithi063.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-09T20:06:10.833 INFO:teuthology.orchestra.run.smithi063.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-09T20:06:10.834 INFO:teuthology.orchestra.run.smithi063.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T20:06:10.834 INFO:teuthology.orchestra.run.smithi063.stdout:The following additional packages will be installed: 2024-05-09T20:06:10.834 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-05-09T20:06:10.835 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-05-09T20:06:10.835 INFO:teuthology.orchestra.run.smithi063.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-05-09T20:06:10.835 INFO:teuthology.orchestra.run.smithi063.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-09T20:06:10.836 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-05-09T20:06:10.837 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-05-09T20:06:10.837 INFO:teuthology.orchestra.run.smithi063.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-05-09T20:06:10.837 INFO:teuthology.orchestra.run.smithi063.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-05-09T20:06:10.837 INFO:teuthology.orchestra.run.smithi063.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-05-09T20:06:10.837 INFO:teuthology.orchestra.run.smithi063.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-05-09T20:06:10.837 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-05-09T20:06:10.837 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet zip 2024-05-09T20:06:10.837 INFO:teuthology.orchestra.run.smithi063.stdout:Suggested packages: 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout:Recommended packages: 2024-05-09T20:06:10.838 INFO:teuthology.orchestra.run.smithi063.stdout: btrfs-tools 2024-05-09T20:06:10.917 INFO:teuthology.orchestra.run.smithi120.stdout:0 upgraded, 118 newly installed, 0 to remove and 307 not upgraded. 2024-05-09T20:06:10.918 INFO:teuthology.orchestra.run.smithi120.stdout:Need to get 181 MB of archives. 2024-05-09T20:06:10.918 INFO:teuthology.orchestra.run.smithi120.stdout:After this operation, 829 MB of additional disk space will be used. 2024-05-09T20:06:10.918 INFO:teuthology.orchestra.run.smithi120.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3307-g04969094-1jammy [3,556 kB] 2024-05-09T20:06:10.940 INFO:teuthology.orchestra.run.smithi063.stdout:The following NEW packages will be installed: 2024-05-09T20:06:10.941 INFO:teuthology.orchestra.run.smithi063.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-05-09T20:06:10.941 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-05-09T20:06:10.941 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-05-09T20:06:10.941 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-05-09T20:06:10.941 INFO:teuthology.orchestra.run.smithi063.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-05-09T20:06:10.941 INFO:teuthology.orchestra.run.smithi063.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-05-09T20:06:10.942 INFO:teuthology.orchestra.run.smithi063.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-05-09T20:06:10.942 INFO:teuthology.orchestra.run.smithi063.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-05-09T20:06:10.942 INFO:teuthology.orchestra.run.smithi063.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-05-09T20:06:10.942 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-05-09T20:06:10.943 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-05-09T20:06:10.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-05-09T20:06:10.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-05-09T20:06:10.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-05-09T20:06:10.944 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-05-09T20:06:10.944 INFO:teuthology.orchestra.run.smithi063.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-05-09T20:06:10.971 INFO:teuthology.orchestra.run.smithi120.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-09T20:06:11.084 INFO:teuthology.orchestra.run.smithi063.stdout:0 upgraded, 118 newly installed, 0 to remove and 307 not upgraded. 2024-05-09T20:06:11.084 INFO:teuthology.orchestra.run.smithi063.stdout:Need to get 181 MB of archives. 2024-05-09T20:06:11.085 INFO:teuthology.orchestra.run.smithi063.stdout:After this operation, 829 MB of additional disk space will be used. 2024-05-09T20:06:11.085 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-05-09T20:06:11.194 INFO:teuthology.orchestra.run.smithi120.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3307-g04969094-1jammy [3,224 kB] 2024-05-09T20:06:11.354 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3307-g04969094-1jammy [712 kB] 2024-05-09T20:06:11.355 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3307-g04969094-1jammy [3,556 kB] 2024-05-09T20:06:11.386 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-09T20:06:11.390 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-09T20:06:11.392 INFO:teuthology.orchestra.run.smithi120.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3307-g04969094-1jammy [356 kB] 2024-05-09T20:06:11.398 INFO:teuthology.orchestra.run.smithi120.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-05-09T20:06:11.411 INFO:teuthology.orchestra.run.smithi120.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-05-09T20:06:11.413 INFO:teuthology.orchestra.run.smithi120.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3307-g04969094-1jammy [32.8 kB] 2024-05-09T20:06:11.414 INFO:teuthology.orchestra.run.smithi120.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3307-g04969094-1jammy [180 kB] 2024-05-09T20:06:11.424 INFO:teuthology.orchestra.run.smithi120.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3307-g04969094-1jammy [67.0 kB] 2024-05-09T20:06:11.428 INFO:teuthology.orchestra.run.smithi120.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3307-g04969094-1jammy [330 kB] 2024-05-09T20:06:11.449 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-09T20:06:11.450 INFO:teuthology.orchestra.run.smithi120.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3307-g04969094-1jammy [6,892 kB] 2024-05-09T20:06:11.491 INFO:teuthology.orchestra.run.smithi120.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-05-09T20:06:11.632 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-09T20:06:11.640 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-09T20:06:11.665 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-09T20:06:11.675 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-09T20:06:11.676 INFO:teuthology.orchestra.run.smithi063.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-09T20:06:11.678 INFO:teuthology.orchestra.run.smithi063.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-09T20:06:11.693 INFO:teuthology.orchestra.run.smithi063.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-09T20:06:11.697 INFO:teuthology.orchestra.run.smithi063.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-09T20:06:11.701 INFO:teuthology.orchestra.run.smithi063.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-09T20:06:11.703 INFO:teuthology.orchestra.run.smithi063.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-09T20:06:11.717 INFO:teuthology.orchestra.run.smithi063.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-09T20:06:11.721 INFO:teuthology.orchestra.run.smithi063.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-09T20:06:11.722 INFO:teuthology.orchestra.run.smithi120.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-05-09T20:06:11.725 INFO:teuthology.orchestra.run.smithi063.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-09T20:06:11.729 INFO:teuthology.orchestra.run.smithi063.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-09T20:06:11.730 INFO:teuthology.orchestra.run.smithi063.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-09T20:06:11.733 INFO:teuthology.orchestra.run.smithi120.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-05-09T20:06:11.745 INFO:teuthology.orchestra.run.smithi063.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-09T20:06:11.747 INFO:teuthology.orchestra.run.smithi063.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-09T20:06:11.747 INFO:teuthology.orchestra.run.smithi063.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-09T20:06:11.748 INFO:teuthology.orchestra.run.smithi063.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-09T20:06:11.748 INFO:teuthology.orchestra.run.smithi063.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-09T20:06:11.763 INFO:teuthology.orchestra.run.smithi120.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-05-09T20:06:11.772 INFO:teuthology.orchestra.run.smithi120.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-05-09T20:06:11.773 INFO:teuthology.orchestra.run.smithi120.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-05-09T20:06:11.775 INFO:teuthology.orchestra.run.smithi120.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-05-09T20:06:11.790 INFO:teuthology.orchestra.run.smithi063.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-09T20:06:11.791 INFO:teuthology.orchestra.run.smithi063.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-09T20:06:11.791 INFO:teuthology.orchestra.run.smithi063.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-09T20:06:11.793 INFO:teuthology.orchestra.run.smithi120.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-05-09T20:06:11.797 INFO:teuthology.orchestra.run.smithi063.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-09T20:06:11.798 INFO:teuthology.orchestra.run.smithi063.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-09T20:06:11.802 INFO:teuthology.orchestra.run.smithi120.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3307-g04969094-1jammy [112 kB] 2024-05-09T20:06:11.806 INFO:teuthology.orchestra.run.smithi120.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3307-g04969094-1jammy [466 kB] 2024-05-09T20:06:11.828 INFO:teuthology.orchestra.run.smithi120.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3307-g04969094-1jammy [25.8 MB] 2024-05-09T20:06:11.832 INFO:teuthology.orchestra.run.smithi063.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-09T20:06:11.832 INFO:teuthology.orchestra.run.smithi063.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-09T20:06:11.834 INFO:teuthology.orchestra.run.smithi063.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-09T20:06:11.835 INFO:teuthology.orchestra.run.smithi063.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-09T20:06:11.835 INFO:teuthology.orchestra.run.smithi063.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-09T20:06:11.859 INFO:teuthology.orchestra.run.smithi063.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-09T20:06:11.860 INFO:teuthology.orchestra.run.smithi063.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-09T20:06:11.861 INFO:teuthology.orchestra.run.smithi063.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-09T20:06:11.871 INFO:teuthology.orchestra.run.smithi063.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-09T20:06:11.872 INFO:teuthology.orchestra.run.smithi063.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-09T20:06:11.885 INFO:teuthology.orchestra.run.smithi063.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-09T20:06:11.886 INFO:teuthology.orchestra.run.smithi063.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-09T20:06:11.889 INFO:teuthology.orchestra.run.smithi063.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-09T20:06:11.893 INFO:teuthology.orchestra.run.smithi063.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-09T20:06:11.894 INFO:teuthology.orchestra.run.smithi063.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-09T20:06:11.912 INFO:teuthology.orchestra.run.smithi063.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-09T20:06:11.913 INFO:teuthology.orchestra.run.smithi063.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-09T20:06:11.918 INFO:teuthology.orchestra.run.smithi063.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-09T20:06:11.919 INFO:teuthology.orchestra.run.smithi063.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-09T20:06:11.921 INFO:teuthology.orchestra.run.smithi063.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-09T20:06:11.940 INFO:teuthology.orchestra.run.smithi063.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-09T20:06:11.961 INFO:teuthology.orchestra.run.smithi120.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-05-09T20:06:12.028 INFO:teuthology.orchestra.run.smithi063.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-09T20:06:12.030 INFO:teuthology.orchestra.run.smithi063.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-09T20:06:12.032 INFO:teuthology.orchestra.run.smithi063.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-09T20:06:12.032 INFO:teuthology.orchestra.run.smithi063.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-09T20:06:12.035 INFO:teuthology.orchestra.run.smithi063.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-09T20:06:12.065 INFO:teuthology.orchestra.run.smithi063.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-09T20:06:12.067 INFO:teuthology.orchestra.run.smithi063.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-09T20:06:12.068 INFO:teuthology.orchestra.run.smithi063.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-09T20:06:12.101 INFO:teuthology.orchestra.run.smithi063.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-09T20:06:12.101 INFO:teuthology.orchestra.run.smithi063.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-09T20:06:12.115 INFO:teuthology.orchestra.run.smithi063.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-09T20:06:12.115 INFO:teuthology.orchestra.run.smithi063.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-09T20:06:12.116 INFO:teuthology.orchestra.run.smithi063.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-09T20:06:12.117 INFO:teuthology.orchestra.run.smithi063.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-09T20:06:12.117 INFO:teuthology.orchestra.run.smithi063.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-09T20:06:12.118 INFO:teuthology.orchestra.run.smithi063.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-09T20:06:12.123 INFO:teuthology.orchestra.run.smithi063.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-09T20:06:12.142 INFO:teuthology.orchestra.run.smithi063.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-09T20:06:12.144 INFO:teuthology.orchestra.run.smithi063.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-09T20:06:12.145 INFO:teuthology.orchestra.run.smithi063.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-09T20:06:12.169 INFO:teuthology.orchestra.run.smithi063.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-09T20:06:12.173 INFO:teuthology.orchestra.run.smithi063.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-09T20:06:12.174 INFO:teuthology.orchestra.run.smithi063.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-09T20:06:12.174 INFO:teuthology.orchestra.run.smithi063.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-09T20:06:12.181 INFO:teuthology.orchestra.run.smithi063.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-09T20:06:12.182 INFO:teuthology.orchestra.run.smithi063.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-09T20:06:12.186 INFO:teuthology.orchestra.run.smithi063.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-09T20:06:12.201 INFO:teuthology.orchestra.run.smithi063.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-09T20:06:12.202 INFO:teuthology.orchestra.run.smithi063.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-09T20:06:12.202 INFO:teuthology.orchestra.run.smithi063.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-09T20:06:12.227 INFO:teuthology.orchestra.run.smithi063.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-09T20:06:12.229 INFO:teuthology.orchestra.run.smithi063.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-09T20:06:12.240 INFO:teuthology.orchestra.run.smithi063.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-09T20:06:12.240 INFO:teuthology.orchestra.run.smithi063.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-09T20:06:12.240 INFO:teuthology.orchestra.run.smithi063.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-09T20:06:12.243 INFO:teuthology.orchestra.run.smithi063.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-09T20:06:12.244 INFO:teuthology.orchestra.run.smithi063.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-09T20:06:12.361 INFO:teuthology.orchestra.run.smithi120.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-05-09T20:06:12.396 INFO:teuthology.orchestra.run.smithi120.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-05-09T20:06:12.398 INFO:teuthology.orchestra.run.smithi120.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-05-09T20:06:12.477 INFO:teuthology.orchestra.run.smithi120.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-05-09T20:06:12.529 INFO:teuthology.orchestra.run.smithi120.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-05-09T20:06:12.550 INFO:teuthology.orchestra.run.smithi120.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-05-09T20:06:12.556 INFO:teuthology.orchestra.run.smithi120.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-05-09T20:06:12.558 INFO:teuthology.orchestra.run.smithi120.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-05-09T20:06:12.569 INFO:teuthology.orchestra.run.smithi120.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-05-09T20:06:12.569 INFO:teuthology.orchestra.run.smithi120.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-05-09T20:06:12.736 INFO:teuthology.orchestra.run.smithi120.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-05-09T20:06:12.819 INFO:teuthology.orchestra.run.smithi120.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-05-09T20:06:12.847 INFO:teuthology.orchestra.run.smithi120.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-05-09T20:06:12.864 INFO:teuthology.orchestra.run.smithi120.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-05-09T20:06:12.909 INFO:teuthology.orchestra.run.smithi120.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-05-09T20:06:13.125 INFO:teuthology.orchestra.run.smithi120.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-05-09T20:06:13.196 INFO:teuthology.orchestra.run.smithi120.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-05-09T20:06:13.196 INFO:teuthology.orchestra.run.smithi120.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-05-09T20:06:13.197 INFO:teuthology.orchestra.run.smithi120.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-05-09T20:06:13.246 INFO:teuthology.orchestra.run.smithi120.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-05-09T20:06:13.246 INFO:teuthology.orchestra.run.smithi120.stdout:Get:45 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3307-g04969094-1jammy [5,095 kB] 2024-05-09T20:06:13.399 INFO:teuthology.orchestra.run.smithi120.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3307-g04969094-1jammy [244 kB] 2024-05-09T20:06:13.406 INFO:teuthology.orchestra.run.smithi120.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3307-g04969094-1jammy [124 kB] 2024-05-09T20:06:13.410 INFO:teuthology.orchestra.run.smithi120.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3307-g04969094-1jammy [1,544 kB] 2024-05-09T20:06:13.411 INFO:teuthology.orchestra.run.smithi120.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-05-09T20:06:13.411 INFO:teuthology.orchestra.run.smithi120.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-05-09T20:06:13.492 INFO:teuthology.orchestra.run.smithi120.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3307-g04969094-1jammy [6,257 kB] 2024-05-09T20:06:13.661 INFO:teuthology.orchestra.run.smithi120.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-05-09T20:06:13.708 INFO:teuthology.orchestra.run.smithi120.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-05-09T20:06:13.721 INFO:teuthology.orchestra.run.smithi120.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-05-09T20:06:13.797 INFO:teuthology.orchestra.run.smithi120.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3307-g04969094-1jammy [22.8 MB] 2024-05-09T20:06:13.890 INFO:teuthology.orchestra.run.smithi120.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-05-09T20:06:13.890 INFO:teuthology.orchestra.run.smithi120.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-05-09T20:06:13.896 INFO:teuthology.orchestra.run.smithi120.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-05-09T20:06:13.901 INFO:teuthology.orchestra.run.smithi120.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-05-09T20:06:13.915 INFO:teuthology.orchestra.run.smithi120.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-05-09T20:06:13.936 INFO:teuthology.orchestra.run.smithi120.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-05-09T20:06:14.106 INFO:teuthology.orchestra.run.smithi120.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-05-09T20:06:14.398 INFO:teuthology.orchestra.run.smithi063.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3307-g04969094-1jammy [3,224 kB] 2024-05-09T20:06:14.450 INFO:teuthology.orchestra.run.smithi120.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-05-09T20:06:14.477 INFO:teuthology.orchestra.run.smithi120.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-05-09T20:06:14.600 INFO:teuthology.orchestra.run.smithi120.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-05-09T20:06:14.616 INFO:teuthology.orchestra.run.smithi120.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-05-09T20:06:14.628 INFO:teuthology.orchestra.run.smithi120.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-05-09T20:06:14.676 INFO:teuthology.orchestra.run.smithi120.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-05-09T20:06:14.820 INFO:teuthology.orchestra.run.smithi120.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3307-g04969094-1jammy [14.0 kB] 2024-05-09T20:06:14.821 INFO:teuthology.orchestra.run.smithi120.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3307-g04969094-1jammy [870 kB] 2024-05-09T20:06:14.838 INFO:teuthology.orchestra.run.smithi120.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3307-g04969094-1jammy [2,437 kB] 2024-05-09T20:06:14.879 INFO:teuthology.orchestra.run.smithi120.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3307-g04969094-1jammy [791 kB] 2024-05-09T20:06:14.917 INFO:teuthology.orchestra.run.smithi120.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3307-g04969094-1jammy [148 kB] 2024-05-09T20:06:14.922 INFO:teuthology.orchestra.run.smithi120.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3307-g04969094-1jammy [2,297 kB] 2024-05-09T20:06:14.955 INFO:teuthology.orchestra.run.smithi120.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-05-09T20:06:14.957 INFO:teuthology.orchestra.run.smithi120.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-05-09T20:06:14.959 INFO:teuthology.orchestra.run.smithi120.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-05-09T20:06:14.960 INFO:teuthology.orchestra.run.smithi120.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-05-09T20:06:15.013 INFO:teuthology.orchestra.run.smithi120.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3307-g04969094-1jammy [8,625 kB] 2024-05-09T20:06:15.125 INFO:teuthology.orchestra.run.smithi120.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-05-09T20:06:15.358 INFO:teuthology.orchestra.run.smithi120.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3307-g04969094-1jammy [14.2 kB] 2024-05-09T20:06:15.411 INFO:teuthology.orchestra.run.smithi120.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3307-g04969094-1jammy [51.7 MB] 2024-05-09T20:06:15.830 INFO:teuthology.orchestra.run.smithi120.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-05-09T20:06:15.835 INFO:teuthology.orchestra.run.smithi120.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-05-09T20:06:15.836 INFO:teuthology.orchestra.run.smithi120.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-05-09T20:06:15.910 INFO:teuthology.orchestra.run.smithi120.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-05-09T20:06:15.910 INFO:teuthology.orchestra.run.smithi120.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-05-09T20:06:15.912 INFO:teuthology.orchestra.run.smithi120.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-05-09T20:06:15.914 INFO:teuthology.orchestra.run.smithi120.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-05-09T20:06:15.915 INFO:teuthology.orchestra.run.smithi120.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-05-09T20:06:15.916 INFO:teuthology.orchestra.run.smithi120.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-05-09T20:06:15.917 INFO:teuthology.orchestra.run.smithi120.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-05-09T20:06:16.083 INFO:teuthology.orchestra.run.smithi120.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-05-09T20:06:16.584 INFO:teuthology.orchestra.run.smithi120.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-05-09T20:06:16.611 INFO:teuthology.orchestra.run.smithi120.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-05-09T20:06:16.633 INFO:teuthology.orchestra.run.smithi120.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-05-09T20:06:16.642 INFO:teuthology.orchestra.run.smithi120.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-05-09T20:06:16.721 INFO:teuthology.orchestra.run.smithi120.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-05-09T20:06:16.752 INFO:teuthology.orchestra.run.smithi120.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-05-09T20:06:16.759 INFO:teuthology.orchestra.run.smithi120.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-05-09T20:06:16.761 INFO:teuthology.orchestra.run.smithi120.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-05-09T20:06:16.798 INFO:teuthology.orchestra.run.smithi120.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-05-09T20:06:16.802 INFO:teuthology.orchestra.run.smithi120.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-05-09T20:06:16.990 INFO:teuthology.orchestra.run.smithi120.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-05-09T20:06:17.078 INFO:teuthology.orchestra.run.smithi120.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-05-09T20:06:17.266 INFO:teuthology.orchestra.run.smithi120.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-05-09T20:06:17.267 INFO:teuthology.orchestra.run.smithi120.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-05-09T20:06:17.343 INFO:teuthology.orchestra.run.smithi063.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3307-g04969094-1jammy [712 kB] 2024-05-09T20:06:17.455 INFO:teuthology.orchestra.run.smithi120.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-05-09T20:06:17.456 INFO:teuthology.orchestra.run.smithi120.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3307-g04969094-1jammy [124 kB] 2024-05-09T20:06:17.458 INFO:teuthology.orchestra.run.smithi120.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3307-g04969094-1jammy [38.9 kB] 2024-05-09T20:06:17.459 INFO:teuthology.orchestra.run.smithi120.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3307-g04969094-1jammy [13.6 MB] 2024-05-09T20:06:17.472 INFO:teuthology.orchestra.run.smithi120.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-05-09T20:06:17.712 INFO:teuthology.orchestra.run.smithi063.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3307-g04969094-1jammy [356 kB] 2024-05-09T20:06:17.857 INFO:teuthology.orchestra.run.smithi120.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-05-09T20:06:17.857 INFO:teuthology.orchestra.run.smithi120.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3307-g04969094-1jammy [91.4 kB] 2024-05-09T20:06:17.857 INFO:teuthology.orchestra.run.smithi120.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-05-09T20:06:17.858 INFO:teuthology.orchestra.run.smithi120.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-05-09T20:06:17.893 INFO:teuthology.orchestra.run.smithi063.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3307-g04969094-1jammy [32.8 kB] 2024-05-09T20:06:17.900 INFO:teuthology.orchestra.run.smithi063.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3307-g04969094-1jammy [180 kB] 2024-05-09T20:06:17.901 INFO:teuthology.orchestra.run.smithi120.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-05-09T20:06:17.914 INFO:teuthology.orchestra.run.smithi120.stdout:Get:118 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-05-09T20:06:17.977 INFO:teuthology.orchestra.run.smithi063.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3307-g04969094-1jammy [67.0 kB] 2024-05-09T20:06:17.982 INFO:teuthology.orchestra.run.smithi063.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3307-g04969094-1jammy [330 kB] 2024-05-09T20:06:18.098 INFO:teuthology.orchestra.run.smithi063.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3307-g04969094-1jammy [6,892 kB] 2024-05-09T20:06:18.724 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 181 MB in 7s (24.6 MB/s) 2024-05-09T20:06:18.882 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-09T20:06:18.982 INFO:teuthology.orchestra.run.smithi120.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-05-09T20:06:18.988 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-09T20:06:19.081 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-09T20:06:19.417 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-09T20:06:19.444 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-09T20:06:19.484 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-09T20:06:19.770 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-09T20:06:19.796 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-09T20:06:19.828 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-09T20:06:20.273 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-09T20:06:20.300 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-09T20:06:20.329 INFO:teuthology.orchestra.run.smithi063.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3307-g04969094-1jammy [112 kB] 2024-05-09T20:06:20.348 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:06:20.363 INFO:teuthology.orchestra.run.smithi063.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3307-g04969094-1jammy [466 kB] 2024-05-09T20:06:20.488 INFO:teuthology.orchestra.run.smithi063.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3307-g04969094-1jammy [25.8 MB] 2024-05-09T20:06:20.743 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-09T20:06:20.769 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-09T20:06:20.809 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:06:21.371 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-09T20:06:21.397 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-09T20:06:21.446 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:06:21.765 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-09T20:06:21.781 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-09T20:06:21.815 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-09T20:06:22.118 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package librados2. 2024-05-09T20:06:22.144 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../007-librados2_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:22.185 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking librados2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:22.671 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package librbd1. 2024-05-09T20:06:22.697 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../008-librbd1_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:22.737 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking librbd1 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:23.190 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcephfs2. 2024-05-09T20:06:23.216 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:23.256 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcephfs2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:23.509 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-rados. 2024-05-09T20:06:23.524 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../010-python3-rados_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:23.559 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-rados (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:23.878 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-09T20:06:23.904 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:06:23.945 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-ceph-argparse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:24.189 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-cephfs. 2024-05-09T20:06:24.204 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:24.247 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-cephfs (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:24.548 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-09T20:06:24.568 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:06:24.608 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-ceph-common (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:24.919 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-09T20:06:24.954 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-09T20:06:24.985 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-09T20:06:25.305 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-prettytable. 2024-05-09T20:06:25.332 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-05-09T20:06:25.372 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-09T20:06:25.599 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-rbd. 2024-05-09T20:06:25.625 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:25.658 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-rbd (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:25.994 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-09T20:06:26.020 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-09T20:06:26.060 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-09T20:06:26.346 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-09T20:06:26.372 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-09T20:06:26.459 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-09T20:06:26.764 INFO:teuthology.orchestra.run.smithi063.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3307-g04969094-1jammy [5,095 kB] 2024-05-09T20:06:26.849 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-09T20:06:26.875 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-09T20:06:26.915 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-09T20:06:27.210 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package lua5.1. 2024-05-09T20:06:27.236 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-09T20:06:27.268 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-09T20:06:27.565 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package lua-any. 2024-05-09T20:06:27.591 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-05-09T20:06:27.632 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-09T20:06:27.647 INFO:teuthology.orchestra.run.smithi063.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3307-g04969094-1jammy [244 kB] 2024-05-09T20:06:27.683 INFO:teuthology.orchestra.run.smithi063.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3307-g04969094-1jammy [124 kB] 2024-05-09T20:06:27.702 INFO:teuthology.orchestra.run.smithi063.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3307-g04969094-1jammy [1,544 kB] 2024-05-09T20:06:27.868 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package zip. 2024-05-09T20:06:27.883 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-05-09T20:06:27.918 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking zip (3.0-12build2) ... 2024-05-09T20:06:27.929 INFO:teuthology.orchestra.run.smithi063.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3307-g04969094-1jammy [6,257 kB] 2024-05-09T20:06:28.196 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package unzip. 2024-05-09T20:06:28.211 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-09T20:06:28.262 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-09T20:06:28.650 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package luarocks. 2024-05-09T20:06:28.676 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-09T20:06:28.717 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-09T20:06:28.897 INFO:teuthology.orchestra.run.smithi063.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3307-g04969094-1jammy [22.8 MB] 2024-05-09T20:06:29.075 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package librgw2. 2024-05-09T20:06:29.102 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../025-librgw2_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:29.142 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking librgw2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:29.694 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-rgw. 2024-05-09T20:06:29.717 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:29.746 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-rgw (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:30.048 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-09T20:06:30.074 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-09T20:06:30.114 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-09T20:06:30.417 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libradosstriper1. 2024-05-09T20:06:30.443 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:30.484 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libradosstriper1 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:30.913 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-common. 2024-05-09T20:06:30.939 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../029-ceph-common_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:30.979 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-common (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:32.368 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-base. 2024-05-09T20:06:32.394 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../030-ceph-base_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:32.444 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-base (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:33.006 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-09T20:06:33.032 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-09T20:06:33.073 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-09T20:06:33.352 INFO:teuthology.orchestra.run.smithi063.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3307-g04969094-1jammy [14.0 kB] 2024-05-09T20:06:33.352 INFO:teuthology.orchestra.run.smithi063.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3307-g04969094-1jammy [870 kB] 2024-05-09T20:06:33.376 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-cheroot. 2024-05-09T20:06:33.402 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-09T20:06:33.442 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-09T20:06:33.477 INFO:teuthology.orchestra.run.smithi063.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3307-g04969094-1jammy [2,437 kB] 2024-05-09T20:06:33.729 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-09T20:06:33.744 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-09T20:06:33.779 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-09T20:06:34.002 INFO:teuthology.orchestra.run.smithi063.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3307-g04969094-1jammy [791 kB] 2024-05-09T20:06:34.074 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-09T20:06:34.100 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-09T20:06:34.140 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-09T20:06:34.192 INFO:teuthology.orchestra.run.smithi063.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3307-g04969094-1jammy [148 kB] 2024-05-09T20:06:34.206 INFO:teuthology.orchestra.run.smithi063.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3307-g04969094-1jammy [2,297 kB] 2024-05-09T20:06:34.443 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-09T20:06:34.470 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-09T20:06:34.510 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-09T20:06:34.693 INFO:teuthology.orchestra.run.smithi063.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3307-g04969094-1jammy [8,625 kB] 2024-05-09T20:06:34.780 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-tempora. 2024-05-09T20:06:34.806 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-05-09T20:06:34.838 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-09T20:06:35.133 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-portend. 2024-05-09T20:06:35.159 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-05-09T20:06:35.199 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-09T20:06:35.528 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-09T20:06:35.554 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-09T20:06:35.594 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-09T20:06:35.897 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-09T20:06:35.924 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-09T20:06:35.964 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-09T20:06:36.163 INFO:teuthology.orchestra.run.smithi063.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3307-g04969094-1jammy [14.2 kB] 2024-05-09T20:06:36.168 INFO:teuthology.orchestra.run.smithi063.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3307-g04969094-1jammy [51.7 MB] 2024-05-09T20:06:36.285 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-natsort. 2024-05-09T20:06:36.311 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-05-09T20:06:36.351 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-09T20:06:36.663 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-logutils. 2024-05-09T20:06:36.689 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-05-09T20:06:36.729 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-09T20:06:37.041 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-09T20:06:37.067 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-09T20:06:37.107 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-09T20:06:37.519 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-mako. 2024-05-09T20:06:37.545 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-09T20:06:37.585 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-09T20:06:37.914 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-09T20:06:37.940 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-09T20:06:37.980 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-09T20:06:38.284 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-09T20:06:38.310 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-09T20:06:38.350 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-09T20:06:38.636 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-webob. 2024-05-09T20:06:38.663 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-09T20:06:38.703 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-09T20:06:39.039 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-waitress. 2024-05-09T20:06:39.066 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-09T20:06:39.117 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-09T20:06:39.418 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-tempita. 2024-05-09T20:06:39.444 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-09T20:06:39.484 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-09T20:06:39.787 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-paste. 2024-05-09T20:06:39.814 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-09T20:06:39.854 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-09T20:06:40.149 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-09T20:06:40.175 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-09T20:06:40.224 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-09T20:06:40.519 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-09T20:06:40.545 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-09T20:06:40.585 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-09T20:06:40.889 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-webtest. 2024-05-09T20:06:40.915 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-05-09T20:06:40.955 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-09T20:06:41.225 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pecan. 2024-05-09T20:06:41.252 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-09T20:06:41.283 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-09T20:06:41.596 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-certifi. 2024-05-09T20:06:41.612 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-05-09T20:06:41.655 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-09T20:06:41.975 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-idna. 2024-05-09T20:06:42.002 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-05-09T20:06:42.333 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-09T20:06:43.170 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-urllib3. 2024-05-09T20:06:43.197 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-09T20:06:43.237 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-09T20:06:43.549 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-requests. 2024-05-09T20:06:43.575 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-09T20:06:43.615 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-09T20:06:43.935 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-09T20:06:43.961 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-09T20:06:44.001 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-09T20:06:44.271 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-09T20:06:44.287 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:06:44.329 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:44.719 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-09T20:06:44.745 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:44.785 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:45.030 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-09T20:06:45.058 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-09T20:06:45.098 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-09T20:06:45.399 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-openssl. 2024-05-09T20:06:45.427 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-05-09T20:06:45.466 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-09T20:06:45.804 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr. 2024-05-09T20:06:45.829 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:45.870 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:46.247 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mon. 2024-05-09T20:06:46.274 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:46.305 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mon (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:46.918 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-osd. 2024-05-09T20:06:46.944 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:46.984 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-osd (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:47.039 INFO:teuthology.orchestra.run.smithi063.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3307-g04969094-1jammy [124 kB] 2024-05-09T20:06:47.042 INFO:teuthology.orchestra.run.smithi063.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3307-g04969094-1jammy [38.9 kB] 2024-05-09T20:06:47.043 INFO:teuthology.orchestra.run.smithi063.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3307-g04969094-1jammy [13.6 MB] 2024-05-09T20:06:48.179 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph. 2024-05-09T20:06:48.195 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../066-ceph_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:48.237 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:48.515 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-fuse. 2024-05-09T20:06:48.542 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:48.574 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-fuse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:48.961 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mds. 2024-05-09T20:06:48.987 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:49.027 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mds (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:49.413 INFO:teuthology.orchestra.run.smithi063.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rgw-d4n-long-descriptions/04969094632d400ff0930fc029b1d19b9c187eef/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3307-g04969094-1jammy [91.4 kB] 2024-05-09T20:06:49.422 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package cephadm. 2024-05-09T20:06:49.448 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../069-cephadm_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:49.488 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking cephadm (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:49.833 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-09T20:06:49.860 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-09T20:06:49.891 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-09T20:06:49.992 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 181 MB in 38s (4,709 kB/s) 2024-05-09T20:06:50.159 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-05-09T20:06:50.211 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-09T20:06:50.226 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-09T20:06:50.263 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-05-09T20:06:50.269 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-05-09T20:06:50.269 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-09T20:06:50.357 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-09T20:06:50.694 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-05-09T20:06:50.721 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-05-09T20:06:50.760 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-09T20:06:51.030 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-05-09T20:06:51.057 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-05-09T20:06:51.088 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-09T20:06:51.275 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-babel. 2024-05-09T20:06:51.302 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-09T20:06:51.333 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-09T20:06:51.534 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-05-09T20:06:51.561 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-09T20:06:51.609 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:06:51.636 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jinja2. 2024-05-09T20:06:51.663 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-09T20:06:51.694 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-09T20:06:51.997 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-09T20:06:52.024 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:06:52.053 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-05-09T20:06:52.055 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:52.080 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-09T20:06:52.120 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:06:52.392 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-09T20:06:52.408 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-05-09T20:06:52.442 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-09T20:06:52.549 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-05-09T20:06:52.576 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-05-09T20:06:52.615 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:06:52.729 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-routes. 2024-05-09T20:06:52.756 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-09T20:06:52.795 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-09T20:06:52.960 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-05-09T20:06:52.987 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-05-09T20:06:53.026 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-09T20:06:53.115 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-09T20:06:53.142 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:06:53.182 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:53.346 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librados2. 2024-05-09T20:06:53.373 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../007-librados2_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:53.413 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librados2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:53.816 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librbd1. 2024-05-09T20:06:53.843 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../008-librbd1_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:53.874 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librbd1 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:54.285 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcephfs2. 2024-05-09T20:06:54.312 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:54.351 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcephfs2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:54.438 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-09T20:06:54.453 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-09T20:06:54.496 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-09T20:06:54.646 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rados. 2024-05-09T20:06:54.673 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../010-python3-rados_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:54.712 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rados (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:54.974 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-joblib. 2024-05-09T20:06:54.990 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-05-09T20:06:55.001 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-09T20:06:55.017 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:06:55.041 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-09T20:06:55.048 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-ceph-argparse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:55.284 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cephfs. 2024-05-09T20:06:55.311 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:55.351 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cephfs (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:55.395 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-09T20:06:55.422 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-09T20:06:55.453 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-09T20:06:55.712 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-ceph-common. 2024-05-09T20:06:55.739 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:06:55.739 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-sklearn. 2024-05-09T20:06:55.766 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-09T20:06:55.787 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-ceph-common (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:55.806 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-09T20:06:56.115 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-wcwidth. 2024-05-09T20:06:56.142 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-05-09T20:06:56.182 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-09T20:06:56.468 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-prettytable. 2024-05-09T20:06:56.482 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-09T20:06:56.484 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-05-09T20:06:56.509 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:06:56.518 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-05-09T20:06:56.634 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:56.762 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rbd. 2024-05-09T20:06:56.789 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:06:56.831 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rbd (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:06:57.176 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-05-09T20:06:57.202 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-05-09T20:06:57.242 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-09T20:06:57.519 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-05-09T20:06:57.536 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-05-09T20:06:57.577 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-05-09T20:06:57.588 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-cachetools. 2024-05-09T20:06:57.615 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-05-09T20:06:57.655 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-09T20:06:57.838 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-05-09T20:06:57.854 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-05-09T20:06:57.897 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-09T20:06:57.958 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-09T20:06:57.985 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-09T20:06:58.024 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-09T20:06:58.209 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package lua5.1. 2024-05-09T20:06:58.236 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-05-09T20:06:58.275 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-05-09T20:06:58.553 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-09T20:06:58.581 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-09T20:06:58.594 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package lua-any. 2024-05-09T20:06:58.620 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-09T20:06:58.621 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-05-09T20:06:58.661 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking lua-any (27ubuntu1) ... 2024-05-09T20:06:58.888 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package zip. 2024-05-09T20:06:58.904 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-05-09T20:06:58.914 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-rsa. 2024-05-09T20:06:58.938 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking zip (3.0-12build2) ... 2024-05-09T20:06:58.941 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-05-09T20:06:58.981 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-09T20:06:59.225 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package unzip. 2024-05-09T20:06:59.251 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-05-09T20:06:59.268 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-google-auth. 2024-05-09T20:06:59.284 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-05-09T20:06:59.291 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-05-09T20:06:59.318 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-09T20:06:59.605 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-09T20:06:59.632 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-09T20:06:59.671 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-09T20:06:59.695 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package luarocks. 2024-05-09T20:06:59.722 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-05-09T20:06:59.761 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-05-09T20:06:59.966 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-websocket. 2024-05-09T20:06:59.994 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-05-09T20:07:00.033 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-09T20:07:00.144 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package librgw2. 2024-05-09T20:07:00.159 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../025-librgw2_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:00.202 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking librgw2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:00.345 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-09T20:07:00.372 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-09T20:07:00.411 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-09T20:07:00.763 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rgw. 2024-05-09T20:07:00.790 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:00.829 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rgw (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:01.043 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-09T20:07:01.059 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:07:01.093 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:01.149 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package liboath0:amd64. 2024-05-09T20:07:01.176 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-05-09T20:07:01.215 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-05-09T20:07:01.412 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-09T20:07:01.440 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-09T20:07:01.479 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-09T20:07:01.493 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libradosstriper1. 2024-05-09T20:07:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:01.560 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libradosstriper1 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:01.774 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-09T20:07:01.801 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-09T20:07:01.832 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-09T20:07:01.979 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-common. 2024-05-09T20:07:02.006 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../029-ceph-common_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:02.046 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-common (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:02.110 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package jq. 2024-05-09T20:07:02.138 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-09T20:07:02.176 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-09T20:07:02.429 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package socat. 2024-05-09T20:07:02.445 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-09T20:07:02.488 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-09T20:07:02.799 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package xmlstarlet. 2024-05-09T20:07:02.815 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-09T20:07:02.857 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-09T20:07:03.212 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-test. 2024-05-09T20:07:03.228 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../097-ceph-test_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:03.262 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-test (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:03.418 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-base. 2024-05-09T20:07:03.444 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../030-ceph-base_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:03.573 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-base (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:04.088 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-05-09T20:07:04.115 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-05-09T20:07:04.146 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-05-09T20:07:04.458 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cheroot. 2024-05-09T20:07:04.485 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-05-09T20:07:04.524 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-09T20:07:04.844 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-05-09T20:07:04.871 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-05-09T20:07:04.910 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-05-09T20:07:05.230 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jaraco.text. 2024-05-09T20:07:05.257 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-05-09T20:07:05.305 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-05-09T20:07:05.492 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-volume. 2024-05-09T20:07:05.520 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:07:05.559 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-volume (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:05.566 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-05-09T20:07:05.582 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-05-09T20:07:05.616 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-05-09T20:07:05.863 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-09T20:07:05.890 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:05.910 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-tempora. 2024-05-09T20:07:05.921 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcephfs-dev (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:05.937 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-05-09T20:07:05.976 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-05-09T20:07:06.224 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-09T20:07:06.252 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-09T20:07:06.279 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-portend. 2024-05-09T20:07:06.290 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-09T20:07:06.306 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-05-09T20:07:06.346 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-05-09T20:07:06.623 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-09T20:07:06.648 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-05-09T20:07:06.650 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-05-09T20:07:06.676 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-05-09T20:07:06.689 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-09T20:07:06.723 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-05-09T20:07:06.959 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cherrypy3. 2024-05-09T20:07:06.975 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-05-09T20:07:07.009 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-05-09T20:07:07.018 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package nvme-cli. 2024-05-09T20:07:07.046 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-09T20:07:07.085 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-09T20:07:07.380 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-natsort. 2024-05-09T20:07:07.407 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-05-09T20:07:07.446 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-05-09T20:07:07.465 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package pkg-config. 2024-05-09T20:07:07.493 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-09T20:07:07.532 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-09T20:07:07.749 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-logutils. 2024-05-09T20:07:07.776 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-05-09T20:07:07.802 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-09T20:07:07.816 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-05-09T20:07:07.830 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-09T20:07:07.868 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-09T20:07:08.085 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-markupsafe. 2024-05-09T20:07:08.112 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-05-09T20:07:08.143 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-05-09T20:07:08.289 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-09T20:07:08.316 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-09T20:07:08.346 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-09T20:07:08.438 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-mako. 2024-05-09T20:07:08.464 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-05-09T20:07:08.504 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-09T20:07:08.591 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pastescript. 2024-05-09T20:07:08.607 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-05-09T20:07:08.641 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-09T20:07:08.824 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-simplegeneric. 2024-05-09T20:07:08.851 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-05-09T20:07:08.890 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-05-09T20:07:08.937 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pluggy. 2024-05-09T20:07:08.964 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-09T20:07:09.003 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-09T20:07:09.160 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-singledispatch. 2024-05-09T20:07:09.187 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-05-09T20:07:09.226 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-05-09T20:07:09.315 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-psutil. 2024-05-09T20:07:09.342 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-09T20:07:09.381 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-09T20:07:09.529 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-webob. 2024-05-09T20:07:09.556 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-05-09T20:07:09.595 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-05-09T20:07:09.718 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-py. 2024-05-09T20:07:09.745 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-05-09T20:07:09.784 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-09T20:07:09.940 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-waitress. 2024-05-09T20:07:09.967 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-05-09T20:07:10.010 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-09T20:07:10.105 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pygments. 2024-05-09T20:07:10.132 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-09T20:07:10.171 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-09T20:07:10.301 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-tempita. 2024-05-09T20:07:10.328 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-05-09T20:07:10.367 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-05-09T20:07:10.600 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-09T20:07:10.616 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-09T20:07:10.650 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-09T20:07:10.670 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-paste. 2024-05-09T20:07:10.697 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-05-09T20:07:10.737 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-05-09T20:07:10.962 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-toml. 2024-05-09T20:07:10.989 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-05-09T20:07:11.028 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-09T20:07:11.073 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-05-09T20:07:11.100 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-05-09T20:07:11.139 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-05-09T20:07:11.356 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pytest. 2024-05-09T20:07:11.384 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-09T20:07:11.422 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-09T20:07:11.434 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pastedeploy. 2024-05-09T20:07:11.461 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-05-09T20:07:11.492 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-05-09T20:07:11.752 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-simplejson. 2024-05-09T20:07:11.770 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-webtest. 2024-05-09T20:07:11.779 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-09T20:07:11.797 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-05-09T20:07:11.818 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-09T20:07:11.837 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-05-09T20:07:12.096 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-09T20:07:12.124 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-09T20:07:12.148 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pecan. 2024-05-09T20:07:12.162 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-09T20:07:12.175 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-05-09T20:07:12.214 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-05-09T20:07:12.561 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-certifi. 2024-05-09T20:07:12.576 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-05-09T20:07:12.619 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-05-09T20:07:12.850 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package radosgw. 2024-05-09T20:07:12.878 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../116-radosgw_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:12.908 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking radosgw (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:12.946 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-idna. 2024-05-09T20:07:12.973 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-05-09T20:07:13.013 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-idna (3.3-1) ... 2024-05-09T20:07:13.324 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-urllib3. 2024-05-09T20:07:13.351 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-05-09T20:07:13.399 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-09T20:07:13.702 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-requests. 2024-05-09T20:07:13.718 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-05-09T20:07:13.720 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package rbd-fuse. 2024-05-09T20:07:13.735 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:13.760 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-09T20:07:13.887 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking rbd-fuse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:14.038 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-werkzeug. 2024-05-09T20:07:14.065 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-05-09T20:07:14.104 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-09T20:07:14.291 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-09T20:07:14.391 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-05-09T20:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:07:14.449 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:14.542 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-09T20:07:14.660 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-09T20:07:14.862 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-05-09T20:07:14.889 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:14.914 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-09T20:07:14.928 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:15.218 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-09T20:07:15.231 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-bcrypt. 2024-05-09T20:07:15.258 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-05-09T20:07:15.297 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-05-09T20:07:15.568 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-openssl. 2024-05-09T20:07:15.594 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-05-09T20:07:15.634 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-05-09T20:07:15.665 INFO:teuthology.orchestra.run.smithi120.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-09T20:07:15.753 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up cephadm (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:15.970 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr. 2024-05-09T20:07:15.997 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:16.036 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:16.286 INFO:teuthology.orchestra.run.smithi120.stdout:Adding system user cephadm....done 2024-05-09T20:07:16.401 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-09T20:07:16.431 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mon. 2024-05-09T20:07:16.458 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:16.497 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mon (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:16.653 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-09T20:07:16.904 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-09T20:07:17.030 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-09T20:07:17.068 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-osd. 2024-05-09T20:07:17.095 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:17.126 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-osd (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:17.307 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-09T20:07:17.566 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-09T20:07:17.676 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-09T20:07:17.961 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-09T20:07:18.321 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-09T20:07:18.329 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph. 2024-05-09T20:07:18.345 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../066-ceph_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:18.387 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:18.598 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-09T20:07:18.715 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-fuse. 2024-05-09T20:07:18.742 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:18.781 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-fuse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:18.882 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-09T20:07:19.151 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mds. 2024-05-09T20:07:19.178 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:19.217 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mds (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:19.334 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-09T20:07:19.570 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package cephadm. 2024-05-09T20:07:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-09T20:07:19.597 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../069-cephadm_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:19.628 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking cephadm (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:19.853 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-09T20:07:19.931 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-asyncssh. 2024-05-09T20:07:19.958 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-05-09T20:07:19.997 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-05-09T20:07:20.104 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-ceph-argparse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:20.309 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-babel-localedata. 2024-05-09T20:07:20.325 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-05-09T20:07:20.364 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-09T20:07:20.366 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-09T20:07:20.490 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-09T20:07:20.616 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-09T20:07:20.717 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-09T20:07:20.951 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-09T20:07:21.355 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-babel. 2024-05-09T20:07:21.382 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-05-09T20:07:21.413 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-05-09T20:07:21.427 INFO:teuthology.orchestra.run.smithi120.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-09T20:07:21.457 INFO:teuthology.orchestra.run.smithi120.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-09T20:07:21.522 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-09T20:07:21.648 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-09T20:07:21.682 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-jinja2. 2024-05-09T20:07:21.710 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-05-09T20:07:21.740 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-09T20:07:22.016 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-09T20:07:22.044 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-05-09T20:07:22.071 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:07:22.110 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:22.117 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-09T20:07:22.360 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-09T20:07:22.427 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-repoze.lru. 2024-05-09T20:07:22.455 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-05-09T20:07:22.488 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-05-09T20:07:22.611 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-09T20:07:22.790 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-routes. 2024-05-09T20:07:22.818 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-05-09T20:07:22.857 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-05-09T20:07:22.921 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up zip (3.0-12build2) ... 2024-05-09T20:07:23.047 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-09T20:07:23.168 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-05-09T20:07:23.196 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:07:23.235 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:23.757 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-09T20:07:24.008 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-09T20:07:24.109 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-09T20:07:24.219 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-09T20:07:24.373 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-05-09T20:07:24.389 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-05-09T20:07:24.431 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-09T20:07:24.537 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-09T20:07:24.901 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-joblib. 2024-05-09T20:07:24.921 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-09T20:07:24.929 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-05-09T20:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-05-09T20:07:25.172 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-09T20:07:25.329 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-05-09T20:07:25.357 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-05-09T20:07:25.395 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-05-09T20:07:25.599 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-09T20:07:25.698 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-sklearn. 2024-05-09T20:07:25.726 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-05-09T20:07:25.764 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-09T20:07:25.884 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-09T20:07:26.227 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-09T20:07:26.465 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-05-09T20:07:26.478 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-09T20:07:26.493 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:07:26.604 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-ceph-common (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:26.616 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:26.897 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-idna (3.3-1) ... 2024-05-09T20:07:27.215 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-09T20:07:27.528 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-cachetools. 2024-05-09T20:07:27.556 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-05-09T20:07:27.594 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-05-09T20:07:27.897 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyasn1. 2024-05-09T20:07:27.925 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-05-09T20:07:27.964 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-05-09T20:07:28.338 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-09T20:07:28.402 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-05-09T20:07:28.430 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-05-09T20:07:28.542 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-05-09T20:07:28.590 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:07:28.725 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-09T20:07:28.853 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-rsa. 2024-05-09T20:07:28.881 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-05-09T20:07:28.919 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-rsa (4.8-1) ... 2024-05-09T20:07:29.076 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-09T20:07:29.210 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-09T20:07:29.239 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-google-auth. 2024-05-09T20:07:29.264 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-05-09T20:07:29.298 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-05-09T20:07:29.353 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-09T20:07:29.593 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-05-09T20:07:29.620 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-05-09T20:07:29.646 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-09T20:07:29.659 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-09T20:07:29.897 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-09T20:07:29.970 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-websocket. 2024-05-09T20:07:29.998 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-05-09T20:07:30.037 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-05-09T20:07:30.148 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:07:30.266 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-09T20:07:30.382 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-kubernetes. 2024-05-09T20:07:30.399 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-05-09T20:07:30.440 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-09T20:07:30.551 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-09T20:07:30.786 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-09T20:07:31.045 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-09T20:07:31.078 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-05-09T20:07:31.095 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:07:31.136 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:31.296 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-09T20:07:31.398 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libonig5:amd64. 2024-05-09T20:07:31.426 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-05-09T20:07:31.455 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-09T20:07:31.547 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-09T20:07:31.784 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjq1:amd64. 2024-05-09T20:07:31.807 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-09T20:07:31.811 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-05-09T20:07:31.858 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-09T20:07:31.925 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-09T20:07:32.119 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package jq. 2024-05-09T20:07:32.136 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-05-09T20:07:32.178 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-05-09T20:07:32.184 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-09T20:07:32.302 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-09T20:07:32.430 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package socat. 2024-05-09T20:07:32.446 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-05-09T20:07:32.488 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-05-09T20:07:32.595 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-09T20:07:32.750 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package xmlstarlet. 2024-05-09T20:07:32.767 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-05-09T20:07:32.799 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-05-09T20:07:32.897 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-09T20:07:33.114 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-test. 2024-05-09T20:07:33.131 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../097-ceph-test_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:33.147 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-09T20:07:33.163 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-test (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:33.274 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-09T20:07:33.391 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-09T20:07:33.464 INFO:teuthology.orchestra.run.smithi120.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-09T20:07:33.718 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-09T20:07:33.853 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-09T20:07:34.255 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-09T20:07:34.514 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-09T20:07:34.640 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-09T20:07:34.891 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:07:34.992 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-09T20:07:35.285 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-09T20:07:35.434 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package ceph-volume. 2024-05-09T20:07:35.463 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-3307-g04969094-1jammy_all.deb ... 2024-05-09T20:07:35.501 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking ceph-volume (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:35.603 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-09T20:07:35.788 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcephfs-dev. 2024-05-09T20:07:35.804 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:35.846 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcephfs-dev (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:35.904 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-09T20:07:36.030 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-09T20:07:36.140 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package lua-socket:amd64. 2024-05-09T20:07:36.168 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-05-09T20:07:36.207 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-09T20:07:36.320 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-09T20:07:36.536 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package lua-sec:amd64. 2024-05-09T20:07:36.564 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-05-09T20:07:36.582 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-09T20:07:36.602 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-05-09T20:07:36.911 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-09T20:07:36.940 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package nvme-cli. 2024-05-09T20:07:36.968 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-05-09T20:07:36.998 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-05-09T20:07:37.288 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-09T20:07:37.411 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package pkg-config. 2024-05-09T20:07:37.439 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-05-09T20:07:37.477 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-05-09T20:07:37.565 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-09T20:07:37.714 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-05-09T20:07:37.730 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-05-09T20:07:37.797 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-05-09T20:07:37.892 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-09T20:07:38.010 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up librados2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:38.145 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:38.208 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-iniconfig. 2024-05-09T20:07:38.236 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-05-09T20:07:38.271 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-09T20:07:38.283 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-05-09T20:07:38.569 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pastescript. 2024-05-09T20:07:38.597 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-05-09T20:07:38.607 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcephfs2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:38.635 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-05-09T20:07:38.709 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libradosstriper1 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:38.810 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-09T20:07:38.938 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pluggy. 2024-05-09T20:07:38.967 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-05-09T20:07:38.996 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-05-09T20:07:39.087 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up librbd1 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:39.213 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-modules-core (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:39.291 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-psutil. 2024-05-09T20:07:39.319 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-05-09T20:07:39.339 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-fuse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:39.357 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-05-09T20:07:39.553 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-09T20:07:39.553 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-09T20:07:39.669 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-py. 2024-05-09T20:07:39.697 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-05-09T20:07:39.735 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-py (1.10.0-1) ... 2024-05-09T20:07:40.055 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pygments. 2024-05-09T20:07:40.057 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcephfs-dev (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:40.084 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-05-09T20:07:40.122 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-05-09T20:07:40.175 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-rados (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:40.276 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up librgw2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:40.377 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-09T20:07:40.551 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pyinotify. 2024-05-09T20:07:40.579 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-05-09T20:07:40.617 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-05-09T20:07:40.919 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-toml. 2024-05-09T20:07:40.947 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-05-09T20:07:40.994 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-05-09T20:07:41.297 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-pytest. 2024-05-09T20:07:41.326 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-05-09T20:07:41.363 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-05-09T20:07:41.598 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-rbd (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:41.724 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up rbd-fuse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:41.734 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-simplejson. 2024-05-09T20:07:41.762 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-05-09T20:07:41.800 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-05-09T20:07:41.825 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-rgw (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:41.951 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-cephfs (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:42.069 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-common (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:42.069 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-05-09T20:07:42.097 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-05-09T20:07:42.135 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-05-09T20:07:42.612 INFO:teuthology.orchestra.run.smithi120.stdout:Adding group ceph....done 2024-05-09T20:07:42.856 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package radosgw. 2024-05-09T20:07:42.884 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../116-radosgw_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:42.922 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking radosgw (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:42.980 INFO:teuthology.orchestra.run.smithi120.stdout:Adding system user ceph....done 2024-05-09T20:07:43.158 INFO:teuthology.orchestra.run.smithi120.stdout:Setting system user ceph properties....done 2024-05-09T20:07:43.172 INFO:teuthology.orchestra.run.smithi120.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-09T20:07:43.310 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-09T20:07:43.668 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-09T20:07:43.717 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package rbd-fuse. 2024-05-09T20:07:43.734 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-3307-g04969094-1jammy_amd64.deb ... 2024-05-09T20:07:43.766 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking rbd-fuse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:44.113 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-test (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:44.238 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-05-09T20:07:44.239 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up radosgw (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:44.488 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-05-09T20:07:44.589 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-05-09T20:07:44.682 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-09T20:07:44.682 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-09T20:07:44.848 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-05-09T20:07:45.149 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-base (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:45.226 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-05-09T20:07:45.462 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-09T20:07:45.668 INFO:teuthology.orchestra.run.smithi063.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-05-09T20:07:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up cephadm (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:45.894 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mds (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:46.113 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-09T20:07:46.114 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-09T20:07:46.182 INFO:teuthology.orchestra.run.smithi063.stdout:Adding system user cephadm....done 2024-05-09T20:07:46.305 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-05-09T20:07:46.579 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:46.581 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-05-09T20:07:46.796 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-09T20:07:46.796 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-09T20:07:46.832 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-05-09T20:07:46.933 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-05-09T20:07:47.192 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-05-09T20:07:47.271 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-osd (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:47.460 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-05-09T20:07:47.548 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-09T20:07:47.548 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-09T20:07:47.587 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-py (1.10.0-1) ... 2024-05-09T20:07:47.896 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-05-09T20:07:48.039 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:48.157 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:48.248 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-05-09T20:07:48.283 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mon (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:48.489 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-09T20:07:48.489 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-09T20:07:48.499 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-05-09T20:07:48.759 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-05-09T20:07:48.993 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-cephadm (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:49.069 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-05-09T20:07:49.144 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:49.270 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-dashboard (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:49.345 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-05-09T20:07:49.421 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-volume (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:49.604 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-05-09T20:07:49.783 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-09T20:07:49.863 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-ceph-argparse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:49.917 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-09T20:07:50.061 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-09T20:07:50.223 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-05-09T20:07:50.357 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-05-09T20:07:50.483 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-05-09T20:07:50.609 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-05-09T20:07:50.868 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-05-09T20:07:51.189 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-09T20:07:51.373 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-05-09T20:07:51.399 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-05-09T20:07:51.463 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-05-09T20:07:51.589 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-05-09T20:07:51.837 INFO:teuthology.orchestra.run.smithi120.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-09T20:07:51.840 DEBUG:teuthology.orchestra.run.smithi120:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-05-09T20:07:51.935 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-05-09T20:07:51.956 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-05-09T20:07:52.083 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-05-09T20:07:52.136 INFO:teuthology.orchestra.run.smithi120.stdout:Building dependency tree... 2024-05-09T20:07:52.137 INFO:teuthology.orchestra.run.smithi120.stdout:Reading state information... 2024-05-09T20:07:52.324 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T20:07:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T20:07:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T20:07:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T20:07:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-09T20:07:52.325 INFO:teuthology.orchestra.run.smithi120.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-09T20:07:52.325 INFO:teuthology.orchestra.run.smithi120.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T20:07:52.326 INFO:teuthology.orchestra.run.smithi120.stdout:The following additional packages will be installed: 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-05-09T20:07:52.327 INFO:teuthology.orchestra.run.smithi120.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-05-09T20:07:52.328 INFO:teuthology.orchestra.run.smithi120.stdout:Suggested packages: 2024-05-09T20:07:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-05-09T20:07:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-05-09T20:07:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-05-09T20:07:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-05-09T20:07:52.328 INFO:teuthology.orchestra.run.smithi120.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-05-09T20:07:52.350 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-05-09T20:07:52.364 INFO:teuthology.orchestra.run.smithi120.stdout:The following NEW packages will be installed: 2024-05-09T20:07:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-05-09T20:07:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-05-09T20:07:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-05-09T20:07:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-05-09T20:07:52.364 INFO:teuthology.orchestra.run.smithi120.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-05-09T20:07:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-05-09T20:07:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-05-09T20:07:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-05-09T20:07:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-05-09T20:07:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-05-09T20:07:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-05-09T20:07:52.365 INFO:teuthology.orchestra.run.smithi120.stdout: s3cmd 2024-05-09T20:07:52.546 INFO:teuthology.orchestra.run.smithi120.stdout:0 upgraded, 35 newly installed, 0 to remove and 307 not upgraded. 2024-05-09T20:07:52.546 INFO:teuthology.orchestra.run.smithi120.stdout:Need to get 10.4 MB of archives. 2024-05-09T20:07:52.546 INFO:teuthology.orchestra.run.smithi120.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-05-09T20:07:52.546 INFO:teuthology.orchestra.run.smithi120.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-05-09T20:07:52.618 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-05-09T20:07:52.628 INFO:teuthology.orchestra.run.smithi120.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-05-09T20:07:52.628 INFO:teuthology.orchestra.run.smithi120.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-05-09T20:07:52.652 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-05-09T20:07:52.822 INFO:teuthology.orchestra.run.smithi120.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-05-09T20:07:52.897 INFO:teuthology.orchestra.run.smithi120.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-05-09T20:07:52.905 INFO:teuthology.orchestra.run.smithi120.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-05-09T20:07:52.927 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up zip (3.0-12build2) ... 2024-05-09T20:07:53.044 INFO:teuthology.orchestra.run.smithi120.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-05-09T20:07:53.053 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-05-09T20:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-05-09T20:07:53.139 INFO:teuthology.orchestra.run.smithi120.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-05-09T20:07:53.142 INFO:teuthology.orchestra.run.smithi120.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-05-09T20:07:53.450 INFO:teuthology.orchestra.run.smithi120.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-05-09T20:07:53.533 INFO:teuthology.orchestra.run.smithi120.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-05-09T20:07:53.805 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-05-09T20:07:54.064 INFO:teuthology.orchestra.run.smithi120.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-05-09T20:07:54.065 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-05-09T20:07:54.066 INFO:teuthology.orchestra.run.smithi120.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-05-09T20:07:54.068 INFO:teuthology.orchestra.run.smithi120.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-05-09T20:07:54.072 INFO:teuthology.orchestra.run.smithi120.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-05-09T20:07:54.073 INFO:teuthology.orchestra.run.smithi120.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-05-09T20:07:54.090 INFO:teuthology.orchestra.run.smithi120.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-05-09T20:07:54.093 INFO:teuthology.orchestra.run.smithi120.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-05-09T20:07:54.124 INFO:teuthology.orchestra.run.smithi120.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2024-05-09T20:07:54.131 INFO:teuthology.orchestra.run.smithi120.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-05-09T20:07:54.165 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-05-09T20:07:54.267 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-05-09T20:07:54.592 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-05-09T20:07:54.985 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-05-09T20:07:55.245 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-05-09T20:07:55.638 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-05-09T20:07:55.931 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-05-09T20:07:56.121 INFO:teuthology.orchestra.run.smithi120.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-05-09T20:07:56.274 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-05-09T20:07:56.385 INFO:teuthology.orchestra.run.smithi120.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-05-09T20:07:56.391 INFO:teuthology.orchestra.run.smithi120.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-05-09T20:07:56.486 INFO:teuthology.orchestra.run.smithi120.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-05-09T20:07:56.516 INFO:teuthology.orchestra.run.smithi120.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-05-09T20:07:56.516 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-05-09T20:07:56.596 INFO:teuthology.orchestra.run.smithi120.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-05-09T20:07:56.634 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-ceph-common (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:07:56.695 INFO:teuthology.orchestra.run.smithi120.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-05-09T20:07:56.724 INFO:teuthology.orchestra.run.smithi120.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-05-09T20:07:56.850 INFO:teuthology.orchestra.run.smithi120.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-05-09T20:07:56.851 INFO:teuthology.orchestra.run.smithi120.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-05-09T20:07:56.918 INFO:teuthology.orchestra.run.smithi120.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-05-09T20:07:56.943 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-idna (3.3-1) ... 2024-05-09T20:07:57.083 INFO:teuthology.orchestra.run.smithi120.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-05-09T20:07:57.166 INFO:teuthology.orchestra.run.smithi120.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-05-09T20:07:57.295 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-05-09T20:07:57.820 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 10.4 MB in 5s (2,068 kB/s) 2024-05-09T20:07:57.903 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libapache-pom-java. 2024-05-09T20:07:57.962 INFO:teuthology.orchestra.run.smithi120.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173897 files and directories currently installed.) 2024-05-09T20:07:57.967 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-05-09T20:07:58.011 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libapache-pom-java (18-1) ... 2024-05-09T20:07:58.272 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-05-09T20:07:58.299 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-05-09T20:07:58.339 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-05-09T20:07:58.550 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-05-09T20:07:58.558 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-05-09T20:07:58.586 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-05-09T20:07:58.625 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-05-09T20:07:58.801 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:07:58.886 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcdi-api-java. 2024-05-09T20:07:58.902 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-05-09T20:07:58.935 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-toml (0.10.2-1) ... 2024-05-09T20:07:58.944 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-05-09T20:07:59.206 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcommons-cli-java. 2024-05-09T20:07:59.220 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-05-09T20:07:59.233 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-05-09T20:07:59.272 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-05-09T20:07:59.508 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcommons-parent-java. 2024-05-09T20:07:59.524 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-05-09T20:07:59.558 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-05-09T20:07:59.613 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-05-09T20:07:59.739 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-05-09T20:07:59.794 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcommons-io-java. 2024-05-09T20:07:59.822 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-05-09T20:07:59.861 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-05-09T20:08:00.048 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-05-09T20:08:00.122 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-05-09T20:08:00.149 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-05-09T20:08:00.188 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-05-09T20:08:00.308 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-05-09T20:08:00.499 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-05-09T20:08:00.527 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-05-09T20:08:00.558 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:08:00.566 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-05-09T20:08:00.684 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-05-09T20:08:00.939 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libjsr305-java. 2024-05-09T20:08:00.967 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-05-09T20:08:00.977 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-05-09T20:08:01.006 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-05-09T20:08:01.262 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-05-09T20:08:01.267 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libguava-java. 2024-05-09T20:08:01.283 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-05-09T20:08:01.325 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libguava-java (29.0-6) ... 2024-05-09T20:08:01.530 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-05-09T20:08:01.727 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libaopalliance-java. 2024-05-09T20:08:01.743 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-05-09T20:08:01.777 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-05-09T20:08:01.789 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-05-09T20:08:02.021 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libguice-java. 2024-05-09T20:08:02.037 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-05-09T20:08:02.040 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-05-09T20:08:02.080 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-05-09T20:08:02.316 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-05-09T20:08:02.433 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-05-09T20:08:02.442 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-05-09T20:08:02.450 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-05-09T20:08:02.492 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-05-09T20:08:02.744 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libjansi-native-java. 2024-05-09T20:08:02.751 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-05-09T20:08:02.772 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-05-09T20:08:02.811 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-05-09T20:08:02.869 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-05-09T20:08:03.046 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libjansi-java. 2024-05-09T20:08:03.062 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-05-09T20:08:03.097 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libjansi-java (1.18-1) ... 2024-05-09T20:08:03.153 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-05-09T20:08:03.315 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libmaven-parent-java. 2024-05-09T20:08:03.331 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-05-09T20:08:03.365 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-05-09T20:08:03.454 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-05-09T20:08:03.617 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-05-09T20:08:03.633 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-05-09T20:08:03.676 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-05-09T20:08:03.714 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-05-09T20:08:03.840 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-05-09T20:08:03.953 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-05-09T20:08:03.965 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-05-09T20:08:03.981 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-05-09T20:08:04.020 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-05-09T20:08:04.055 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-05-09T20:08:04.281 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-05-09T20:08:04.300 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-05-09T20:08:04.309 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-05-09T20:08:04.347 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-05-09T20:08:04.426 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-05-09T20:08:04.659 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-05-09T20:08:04.676 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2024-05-09T20:08:04.718 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-05-09T20:08:04.820 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-05-09T20:08:04.953 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-05-09T20:08:04.969 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-05-09T20:08:05.020 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-05-09T20:08:05.079 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up lua-any (27ubuntu1) ... 2024-05-09T20:08:05.213 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-portend (3.0.0-1) ... 2024-05-09T20:08:05.273 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-05-09T20:08:05.289 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-05-09T20:08:05.331 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-05-09T20:08:05.473 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-05-09T20:08:05.583 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-05-09T20:08:05.598 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-05-09T20:08:05.611 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-05-09T20:08:05.650 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-05-09T20:08:05.903 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-05-09T20:08:05.908 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-05-09T20:08:05.919 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-05-09T20:08:05.953 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-05-09T20:08:06.201 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-05-09T20:08:06.230 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-05-09T20:08:06.246 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-05-09T20:08:06.288 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-05-09T20:08:06.468 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-05-09T20:08:06.516 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libslf4j-java. 2024-05-09T20:08:06.532 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-05-09T20:08:06.569 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rsa (4.8-1) ... 2024-05-09T20:08:06.574 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-05-09T20:08:06.836 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libsisu-inject-java. 2024-05-09T20:08:06.845 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-05-09T20:08:06.864 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-05-09T20:08:06.902 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-05-09T20:08:07.113 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-05-09T20:08:07.197 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-05-09T20:08:07.225 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-05-09T20:08:07.263 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-05-09T20:08:07.466 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-05-09T20:08:07.524 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libmaven3-core-java. 2024-05-09T20:08:07.540 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-05-09T20:08:07.574 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-05-09T20:08:07.843 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-05-09T20:08:07.928 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libwagon-file-java. 2024-05-09T20:08:07.944 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-05-09T20:08:07.978 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-05-09T20:08:08.152 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-05-09T20:08:08.222 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-05-09T20:08:08.238 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-05-09T20:08:08.281 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-05-09T20:08:08.530 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-05-09T20:08:08.656 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librados2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:08.775 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package maven. 2024-05-09T20:08:08.782 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:08.803 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-05-09T20:08:08.841 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking maven (3.6.3-5) ... 2024-05-09T20:08:08.908 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-05-09T20:08:09.111 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-magic. 2024-05-09T20:08:09.127 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-05-09T20:08:09.169 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-05-09T20:08:09.244 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcephfs2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:09.378 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libradosstriper1 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:09.447 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package s3cmd. 2024-05-09T20:08:09.475 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-05-09T20:08:09.496 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-05-09T20:08:09.513 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-05-09T20:08:09.764 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librbd1 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:09.876 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-05-09T20:08:09.890 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-modules-core (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:09.993 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-05-09T20:08:10.016 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-fuse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:10.119 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-05-09T20:08:10.233 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-09T20:08:10.234 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-05-09T20:08:10.245 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-05-09T20:08:10.372 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-05-09T20:08:10.497 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-05-09T20:08:10.623 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-05-09T20:08:10.700 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcephfs-dev (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:10.749 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-05-09T20:08:10.826 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rados (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:10.952 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up librgw2 (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:11.011 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-05-09T20:08:11.078 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-05-09T20:08:11.177 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-05-09T20:08:11.278 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libapache-pom-java (18-1) ... 2024-05-09T20:08:11.379 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-05-09T20:08:11.505 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-05-09T20:08:11.631 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-05-09T20:08:11.757 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-05-09T20:08:11.883 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-05-09T20:08:12.164 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-05-09T20:08:12.293 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-05-09T20:08:12.390 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rbd (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:12.411 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-05-09T20:08:12.516 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up rbd-fuse (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:12.528 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libmaven-parent-java (31-2) ... 2024-05-09T20:08:12.629 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcommons-parent-java (43-1) ... 2024-05-09T20:08:12.642 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-rgw (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:12.756 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-05-09T20:08:12.768 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-cephfs (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:12.882 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up s3cmd (2.2.0-1) ... 2024-05-09T20:08:12.894 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-common (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:13.183 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-05-09T20:08:13.309 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-05-09T20:08:13.395 INFO:teuthology.orchestra.run.smithi063.stdout:Adding group ceph....done 2024-05-09T20:08:13.435 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libguava-java (29.0-6) ... 2024-05-09T20:08:13.561 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-05-09T20:08:13.670 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-05-09T20:08:13.721 INFO:teuthology.orchestra.run.smithi063.stdout:Adding system user ceph....done 2024-05-09T20:08:13.779 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-05-09T20:08:13.890 INFO:teuthology.orchestra.run.smithi063.stdout:Setting system user ceph properties....done 2024-05-09T20:08:13.897 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-05-09T20:08:13.908 INFO:teuthology.orchestra.run.smithi063.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-05-09T20:08:14.023 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libguice-java (4.2.3-2) ... 2024-05-09T20:08:14.050 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-05-09T20:08:14.149 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libjansi-java (1.18-1) ... 2024-05-09T20:08:14.274 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-05-09T20:08:14.388 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-05-09T20:08:14.400 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-05-09T20:08:14.526 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up maven (3.6.3-5) ... 2024-05-09T20:08:14.784 INFO:teuthology.orchestra.run.smithi120.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-05-09T20:08:14.855 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-09T20:08:14.869 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-test (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:15.046 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up radosgw (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:15.544 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-09T20:08:15.545 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-05-09T20:08:16.014 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-base (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:16.349 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-05-09T20:08:16.443 INFO:teuthology.orchestra.run.smithi120.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-09T20:08:16.448 DEBUG:teuthology.parallel:result is None 2024-05-09T20:08:16.816 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mds (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:17.052 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-09T20:08:17.052 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-05-09T20:08:17.559 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:17.786 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-09T20:08:17.786 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-05-09T20:08:18.260 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-osd (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:18.540 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-09T20:08:18.540 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-05-09T20:08:19.045 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:19.146 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:19.272 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mon (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:19.489 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-09T20:08:19.489 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-05-09T20:08:19.990 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-cephadm (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:20.327 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:20.461 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-mgr-dashboard (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:20.604 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up ceph-volume (19.0.0-3307-g04969094-1jammy) ... 2024-05-09T20:08:20.948 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-09T20:08:21.057 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-09T20:08:21.210 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-09T20:08:22.318 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-05-09T20:08:23.015 INFO:teuthology.orchestra.run.smithi063.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-09T20:08:23.018 DEBUG:teuthology.orchestra.run.smithi063:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install s3cmd maven 2024-05-09T20:08:23.081 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-09T20:08:23.287 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-05-09T20:08:23.287 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-05-09T20:08:23.484 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T20:08:23.484 INFO:teuthology.orchestra.run.smithi063.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T20:08:23.484 INFO:teuthology.orchestra.run.smithi063.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T20:08:23.484 INFO:teuthology.orchestra.run.smithi063.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T20:08:23.485 INFO:teuthology.orchestra.run.smithi063.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-09T20:08:23.485 INFO:teuthology.orchestra.run.smithi063.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-09T20:08:23.485 INFO:teuthology.orchestra.run.smithi063.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T20:08:23.486 INFO:teuthology.orchestra.run.smithi063.stdout:The following additional packages will be installed: 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-05-09T20:08:23.487 INFO:teuthology.orchestra.run.smithi063.stdout: libwagon-http-shaded-java libwagon-provider-api-java python3-magic 2024-05-09T20:08:23.488 INFO:teuthology.orchestra.run.smithi063.stdout:Suggested packages: 2024-05-09T20:08:23.488 INFO:teuthology.orchestra.run.smithi063.stdout: libaopalliance-java-doc libatinject-jsr330-api-java-doc libel-api-java 2024-05-09T20:08:23.488 INFO:teuthology.orchestra.run.smithi063.stdout: libcommons-io-java-doc libcommons-lang3-java-doc libasm-java libcglib-java 2024-05-09T20:08:23.488 INFO:teuthology.orchestra.run.smithi063.stdout: libjsr305-java-doc libmaven-shared-utils-java-doc liblogback-java 2024-05-09T20:08:23.488 INFO:teuthology.orchestra.run.smithi063.stdout: libplexus-classworlds-java-doc libplexus-sec-dispatcher-java-doc 2024-05-09T20:08:23.488 INFO:teuthology.orchestra.run.smithi063.stdout: libplexus-utils2-java-doc testng libcommons-logging-java liblog4j1.2-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout:The following NEW packages will be installed: 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libaopalliance-java libapache-pom-java libatinject-jsr330-api-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libcdi-api-java libcommons-cli-java libcommons-io-java libcommons-lang3-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libcommons-parent-java libgeronimo-annotation-1.3-spec-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libgeronimo-interceptor-3.0-spec-java libguava-java libguice-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libhawtjni-runtime-java libjansi-java libjansi-native-java libjsr305-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libmaven-parent-java libmaven-resolver-java libmaven-shared-utils-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libmaven3-core-java libplexus-cipher-java libplexus-classworlds-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libplexus-component-annotations-java libplexus-interpolation-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libplexus-sec-dispatcher-java libplexus-utils2-java libsisu-inject-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libsisu-plexus-java libslf4j-java libwagon-file-java 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: libwagon-http-shaded-java libwagon-provider-api-java maven python3-magic 2024-05-09T20:08:23.531 INFO:teuthology.orchestra.run.smithi063.stdout: s3cmd 2024-05-09T20:08:23.635 INFO:teuthology.orchestra.run.smithi063.stdout:0 upgraded, 35 newly installed, 0 to remove and 307 not upgraded. 2024-05-09T20:08:23.635 INFO:teuthology.orchestra.run.smithi063.stdout:Need to get 10.4 MB of archives. 2024-05-09T20:08:23.635 INFO:teuthology.orchestra.run.smithi063.stdout:After this operation, 13.9 MB of additional disk space will be used. 2024-05-09T20:08:23.636 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libapache-pom-java all 18-1 [4,720 B] 2024-05-09T20:08:23.681 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libatinject-jsr330-api-java all 1.0+ds1-5 [5,348 B] 2024-05-09T20:08:23.682 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-interceptor-3.0-spec-java all 1.0.1-4fakesync [8,616 B] 2024-05-09T20:08:23.690 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcdi-api-java all 1.2-3 [54.3 kB] 2024-05-09T20:08:23.781 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-cli-java all 1.4-2 [55.8 kB] 2024-05-09T20:08:23.828 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-parent-java all 43-1 [10.8 kB] 2024-05-09T20:08:23.835 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-io-java all 2.11.0-2 [297 kB] 2024-05-09T20:08:23.924 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libcommons-lang3-java all 3.11-1 [526 kB] 2024-05-09T20:08:23.987 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libgeronimo-annotation-1.3-spec-java all 1.3-1 [11.2 kB] 2024-05-09T20:08:23.988 INFO:teuthology.orchestra.run.smithi063.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjsr305-java all 0.1~+svn49-11 [27.0 kB] 2024-05-09T20:08:23.992 INFO:teuthology.orchestra.run.smithi063.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguava-java all 29.0-6 [2,418 kB] 2024-05-09T20:08:24.147 INFO:teuthology.orchestra.run.smithi063.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libaopalliance-java all 20070526-6 [9,084 B] 2024-05-09T20:08:24.148 INFO:teuthology.orchestra.run.smithi063.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libguice-java all 4.2.3-2 [1,434 kB] 2024-05-09T20:08:24.172 INFO:teuthology.orchestra.run.smithi063.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libhawtjni-runtime-java all 1.17-1 [28.8 kB] 2024-05-09T20:08:24.173 INFO:teuthology.orchestra.run.smithi063.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-native-java all 1.8-1 [23.8 kB] 2024-05-09T20:08:24.174 INFO:teuthology.orchestra.run.smithi063.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libjansi-java all 1.18-1 [56.8 kB] 2024-05-09T20:08:24.177 INFO:teuthology.orchestra.run.smithi063.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-parent-java all 31-2 [5,140 B] 2024-05-09T20:08:24.177 INFO:teuthology.orchestra.run.smithi063.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-utils2-java all 3.3.0-1 [250 kB] 2024-05-09T20:08:24.179 INFO:teuthology.orchestra.run.smithi063.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-provider-api-java all 3.3.4-1 [48.5 kB] 2024-05-09T20:08:24.180 INFO:teuthology.orchestra.run.smithi063.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven-resolver-java all 1.4.2-3build1 [555 kB] 2024-05-09T20:08:24.195 INFO:teuthology.orchestra.run.smithi063.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libmaven-shared-utils-java all 3.3.0-1ubuntu0.22.04.1 [148 kB] 2024-05-09T20:08:24.236 INFO:teuthology.orchestra.run.smithi063.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-cipher-java all 1.8-2 [15.1 kB] 2024-05-09T20:08:24.237 INFO:teuthology.orchestra.run.smithi063.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-classworlds-java all 2.6.0-1 [49.4 kB] 2024-05-09T20:08:24.238 INFO:teuthology.orchestra.run.smithi063.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-component-annotations-java all 2.1.0-1 [6,564 B] 2024-05-09T20:08:24.239 INFO:teuthology.orchestra.run.smithi063.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-interpolation-java all 1.26-1 [76.8 kB] 2024-05-09T20:08:24.241 INFO:teuthology.orchestra.run.smithi063.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libplexus-sec-dispatcher-java all 1.4-4 [28.1 kB] 2024-05-09T20:08:24.242 INFO:teuthology.orchestra.run.smithi063.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libslf4j-java all 1.7.32-1 [141 kB] 2024-05-09T20:08:24.246 INFO:teuthology.orchestra.run.smithi063.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-inject-java all 0.3.4-2 [347 kB] 2024-05-09T20:08:24.254 INFO:teuthology.orchestra.run.smithi063.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libsisu-plexus-java all 0.3.4-3 [181 kB] 2024-05-09T20:08:24.258 INFO:teuthology.orchestra.run.smithi063.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libmaven3-core-java all 3.6.3-5 [1,535 kB] 2024-05-09T20:08:24.304 INFO:teuthology.orchestra.run.smithi063.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-file-java all 3.3.4-1 [7,928 B] 2024-05-09T20:08:24.304 INFO:teuthology.orchestra.run.smithi063.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libwagon-http-shaded-java all 3.3.4-1 [1,855 kB] 2024-05-09T20:08:24.418 INFO:teuthology.orchestra.run.smithi063.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/universe amd64 maven all 3.6.3-5 [17.8 kB] 2024-05-09T20:08:24.418 INFO:teuthology.orchestra.run.smithi063.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-magic all 2:0.4.24-2 [12.6 kB] 2024-05-09T20:08:24.419 INFO:teuthology.orchestra.run.smithi063.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/universe amd64 s3cmd all 2.2.0-1 [120 kB] 2024-05-09T20:08:24.880 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 10.4 MB in 1s (11.8 MB/s) 2024-05-09T20:08:24.972 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libapache-pom-java. 2024-05-09T20:08:25.031 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173897 files and directories currently installed.) 2024-05-09T20:08:25.036 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../00-libapache-pom-java_18-1_all.deb ... 2024-05-09T20:08:25.080 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libapache-pom-java (18-1) ... 2024-05-09T20:08:25.324 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libatinject-jsr330-api-java. 2024-05-09T20:08:25.341 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../01-libatinject-jsr330-api-java_1.0+ds1-5_all.deb ... 2024-05-09T20:08:25.441 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-05-09T20:08:26.281 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libgeronimo-interceptor-3.0-spec-java. 2024-05-09T20:08:26.310 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../02-libgeronimo-interceptor-3.0-spec-java_1.0.1-4fakesync_all.deb ... 2024-05-09T20:08:26.348 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-05-09T20:08:26.600 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcdi-api-java. 2024-05-09T20:08:26.629 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../03-libcdi-api-java_1.2-3_all.deb ... 2024-05-09T20:08:26.667 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcdi-api-java (1.2-3) ... 2024-05-09T20:08:26.936 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcommons-cli-java. 2024-05-09T20:08:26.965 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../04-libcommons-cli-java_1.4-2_all.deb ... 2024-05-09T20:08:27.003 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcommons-cli-java (1.4-2) ... 2024-05-09T20:08:27.247 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcommons-parent-java. 2024-05-09T20:08:27.276 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../05-libcommons-parent-java_43-1_all.deb ... 2024-05-09T20:08:27.305 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcommons-parent-java (43-1) ... 2024-05-09T20:08:27.558 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcommons-io-java. 2024-05-09T20:08:27.586 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../06-libcommons-io-java_2.11.0-2_all.deb ... 2024-05-09T20:08:27.624 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcommons-io-java (2.11.0-2) ... 2024-05-09T20:08:27.894 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libcommons-lang3-java. 2024-05-09T20:08:27.923 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../07-libcommons-lang3-java_3.11-1_all.deb ... 2024-05-09T20:08:27.960 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libcommons-lang3-java (3.11-1) ... 2024-05-09T20:08:28.254 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libgeronimo-annotation-1.3-spec-java. 2024-05-09T20:08:28.271 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../08-libgeronimo-annotation-1.3-spec-java_1.3-1_all.deb ... 2024-05-09T20:08:28.313 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-05-09T20:08:28.565 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjsr305-java. 2024-05-09T20:08:28.594 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../09-libjsr305-java_0.1~+svn49-11_all.deb ... 2024-05-09T20:08:28.632 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjsr305-java (0.1~+svn49-11) ... 2024-05-09T20:08:28.885 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libguava-java. 2024-05-09T20:08:28.902 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../10-libguava-java_29.0-6_all.deb ... 2024-05-09T20:08:28.943 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libguava-java (29.0-6) ... 2024-05-09T20:08:29.362 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libaopalliance-java. 2024-05-09T20:08:29.379 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../11-libaopalliance-java_20070526-6_all.deb ... 2024-05-09T20:08:29.412 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libaopalliance-java (20070526-6) ... 2024-05-09T20:08:29.656 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libguice-java. 2024-05-09T20:08:29.673 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../12-libguice-java_4.2.3-2_all.deb ... 2024-05-09T20:08:29.715 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libguice-java (4.2.3-2) ... 2024-05-09T20:08:30.085 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libhawtjni-runtime-java. 2024-05-09T20:08:30.102 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../13-libhawtjni-runtime-java_1.17-1_all.deb ... 2024-05-09T20:08:30.144 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libhawtjni-runtime-java (1.17-1) ... 2024-05-09T20:08:30.396 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjansi-native-java. 2024-05-09T20:08:30.413 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../14-libjansi-native-java_1.8-1_all.deb ... 2024-05-09T20:08:30.454 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjansi-native-java (1.8-1) ... 2024-05-09T20:08:30.699 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libjansi-java. 2024-05-09T20:08:30.717 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../15-libjansi-java_1.18-1_all.deb ... 2024-05-09T20:08:30.757 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libjansi-java (1.18-1) ... 2024-05-09T20:08:31.010 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libmaven-parent-java. 2024-05-09T20:08:31.027 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../16-libmaven-parent-java_31-2_all.deb ... 2024-05-09T20:08:31.085 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libmaven-parent-java (31-2) ... 2024-05-09T20:08:31.337 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libplexus-utils2-java. 2024-05-09T20:08:31.366 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../17-libplexus-utils2-java_3.3.0-1_all.deb ... 2024-05-09T20:08:31.403 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libplexus-utils2-java (3.3.0-1) ... 2024-05-09T20:08:31.690 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libwagon-provider-api-java. 2024-05-09T20:08:31.707 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../18-libwagon-provider-api-java_3.3.4-1_all.deb ... 2024-05-09T20:08:31.748 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libwagon-provider-api-java (3.3.4-1) ... 2024-05-09T20:08:31.976 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libmaven-resolver-java. 2024-05-09T20:08:31.994 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../19-libmaven-resolver-java_1.4.2-3build1_all.deb ... 2024-05-09T20:08:32.034 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libmaven-resolver-java (1.4.2-3build1) ... 2024-05-09T20:08:32.321 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libmaven-shared-utils-java. 2024-05-09T20:08:32.338 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../20-libmaven-shared-utils-java_3.3.0-1ubuntu0.22.04.1_all.deb ... 2024-05-09T20:08:32.379 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-05-09T20:08:32.632 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libplexus-cipher-java. 2024-05-09T20:08:32.660 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../21-libplexus-cipher-java_1.8-2_all.deb ... 2024-05-09T20:08:32.699 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libplexus-cipher-java (1.8-2) ... 2024-05-09T20:08:32.935 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libplexus-classworlds-java. 2024-05-09T20:08:32.951 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../22-libplexus-classworlds-java_2.6.0-1_all.deb ... 2024-05-09T20:08:32.985 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libplexus-classworlds-java (2.6.0-1) ... 2024-05-09T20:08:33.212 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libplexus-component-annotations-java. 2024-05-09T20:08:33.229 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../23-libplexus-component-annotations-java_2.1.0-1_all.deb ... 2024-05-09T20:08:33.270 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libplexus-component-annotations-java (2.1.0-1) ... 2024-05-09T20:08:33.531 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libplexus-interpolation-java. 2024-05-09T20:08:33.548 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../24-libplexus-interpolation-java_1.26-1_all.deb ... 2024-05-09T20:08:33.590 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libplexus-interpolation-java (1.26-1) ... 2024-05-09T20:08:33.851 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libplexus-sec-dispatcher-java. 2024-05-09T20:08:33.879 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../25-libplexus-sec-dispatcher-java_1.4-4_all.deb ... 2024-05-09T20:08:33.917 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libplexus-sec-dispatcher-java (1.4-4) ... 2024-05-09T20:08:34.153 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libslf4j-java. 2024-05-09T20:08:34.170 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../26-libslf4j-java_1.7.32-1_all.deb ... 2024-05-09T20:08:34.203 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libslf4j-java (1.7.32-1) ... 2024-05-09T20:08:34.473 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libsisu-inject-java. 2024-05-09T20:08:34.490 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../27-libsisu-inject-java_0.3.4-2_all.deb ... 2024-05-09T20:08:34.532 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libsisu-inject-java (0.3.4-2) ... 2024-05-09T20:08:34.801 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libsisu-plexus-java. 2024-05-09T20:08:34.830 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../28-libsisu-plexus-java_0.3.4-3_all.deb ... 2024-05-09T20:08:34.868 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libsisu-plexus-java (0.3.4-3) ... 2024-05-09T20:08:35.112 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libmaven3-core-java. 2024-05-09T20:08:35.141 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../29-libmaven3-core-java_3.6.3-5_all.deb ... 2024-05-09T20:08:35.179 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libmaven3-core-java (3.6.3-5) ... 2024-05-09T20:08:35.591 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libwagon-file-java. 2024-05-09T20:08:35.608 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../30-libwagon-file-java_3.3.4-1_all.deb ... 2024-05-09T20:08:35.667 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libwagon-file-java (3.3.4-1) ... 2024-05-09T20:08:35.919 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libwagon-http-shaded-java. 2024-05-09T20:08:35.948 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../31-libwagon-http-shaded-java_3.3.4-1_all.deb ... 2024-05-09T20:08:35.986 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libwagon-http-shaded-java (3.3.4-1) ... 2024-05-09T20:08:36.438 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package maven. 2024-05-09T20:08:36.467 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../32-maven_3.6.3-5_all.deb ... 2024-05-09T20:08:36.505 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking maven (3.6.3-5) ... 2024-05-09T20:08:36.800 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package python3-magic. 2024-05-09T20:08:36.829 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../33-python3-magic_2%3a0.4.24-2_all.deb ... 2024-05-09T20:08:36.867 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking python3-magic (2:0.4.24-2) ... 2024-05-09T20:08:37.203 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package s3cmd. 2024-05-09T20:08:37.231 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../34-s3cmd_2.2.0-1_all.deb ... 2024-05-09T20:08:37.269 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking s3cmd (2.2.0-1) ... 2024-05-09T20:08:37.632 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libslf4j-java (1.7.32-1) ... 2024-05-09T20:08:37.750 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libplexus-utils2-java (3.3.0-1) ... 2024-05-09T20:08:37.876 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libplexus-classworlds-java (2.6.0-1) ... 2024-05-09T20:08:38.002 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjsr305-java (0.1~+svn49-11) ... 2024-05-09T20:08:38.128 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libaopalliance-java (20070526-6) ... 2024-05-09T20:08:38.254 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcommons-cli-java (1.4-2) ... 2024-05-09T20:08:38.380 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libplexus-component-annotations-java (2.1.0-1) ... 2024-05-09T20:08:38.506 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libplexus-cipher-java (1.8-2) ... 2024-05-09T20:08:38.632 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libgeronimo-annotation-1.3-spec-java (1.3-1) ... 2024-05-09T20:08:38.733 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libgeronimo-interceptor-3.0-spec-java (1.0.1-4fakesync) ... 2024-05-09T20:08:38.859 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libapache-pom-java (18-1) ... 2024-05-09T20:08:38.985 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libatinject-jsr330-api-java (1.0+ds1-5) ... 2024-05-09T20:08:39.111 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libplexus-interpolation-java (1.26-1) ... 2024-05-09T20:08:39.262 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libplexus-sec-dispatcher-java (1.4-4) ... 2024-05-09T20:08:39.388 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libwagon-http-shaded-java (3.3.4-1) ... 2024-05-09T20:08:39.514 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up python3-magic (2:0.4.24-2) ... 2024-05-09T20:08:39.782 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcdi-api-java (1.2-3) ... 2024-05-09T20:08:39.883 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libhawtjni-runtime-java (1.17-1) ... 2024-05-09T20:08:40.000 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libwagon-provider-api-java (3.3.4-1) ... 2024-05-09T20:08:40.143 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libmaven-parent-java (31-2) ... 2024-05-09T20:08:40.269 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcommons-parent-java (43-1) ... 2024-05-09T20:08:40.395 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libsisu-inject-java (0.3.4-2) ... 2024-05-09T20:08:40.521 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up s3cmd (2.2.0-1) ... 2024-05-09T20:08:40.831 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libsisu-plexus-java (0.3.4-3) ... 2024-05-09T20:08:40.957 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libmaven-resolver-java (1.4.2-3build1) ... 2024-05-09T20:08:41.083 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libguava-java (29.0-6) ... 2024-05-09T20:08:41.209 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcommons-lang3-java (3.11-1) ... 2024-05-09T20:08:41.335 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjansi-native-java (1.8-1) ... 2024-05-09T20:08:41.461 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libwagon-file-java (3.3.4-1) ... 2024-05-09T20:08:41.587 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libcommons-io-java (2.11.0-2) ... 2024-05-09T20:08:41.713 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libguice-java (4.2.3-2) ... 2024-05-09T20:08:41.839 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libjansi-java (1.18-1) ... 2024-05-09T20:08:41.957 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libmaven-shared-utils-java (3.3.0-1ubuntu0.22.04.1) ... 2024-05-09T20:08:42.083 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libmaven3-core-java (3.6.3-5) ... 2024-05-09T20:08:42.209 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up maven (3.6.3-5) ... 2024-05-09T20:08:42.467 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/share/maven/bin/mvn to provide /usr/bin/mvn (mvn) in auto mode 2024-05-09T20:08:42.538 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-09T20:08:43.897 INFO:teuthology.orchestra.run.smithi063.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-05-09T20:08:43.902 DEBUG:teuthology.parallel:result is None 2024-05-09T20:08:43.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=04969094632d400ff0930fc029b1d19b9c187eef 2024-05-09T20:08:44.044 DEBUG:teuthology.orchestra.run.smithi063:> dpkg-query -W -f '${Version}' ceph 2024-05-09T20:08:44.063 INFO:teuthology.orchestra.run.smithi063.stdout:19.0.0-3307-g04969094-1jammy 2024-05-09T20:08:44.063 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3307-g04969094-1jammy 2024-05-09T20:08:44.063 INFO:teuthology.task.install:The correct ceph version 19.0.0-3307-g04969094-1jammy is installed. 2024-05-09T20:08:44.065 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=04969094632d400ff0930fc029b1d19b9c187eef 2024-05-09T20:08:44.287 DEBUG:teuthology.orchestra.run.smithi120:> dpkg-query -W -f '${Version}' ceph 2024-05-09T20:08:44.307 INFO:teuthology.orchestra.run.smithi120.stdout:19.0.0-3307-g04969094-1jammy 2024-05-09T20:08:44.307 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3307-g04969094-1jammy 2024-05-09T20:08:44.307 INFO:teuthology.task.install:The correct ceph version 19.0.0-3307-g04969094-1jammy is installed. 2024-05-09T20:08:44.309 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T20:08:44.309 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:44.309 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T20:08:44.324 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:08:44.324 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T20:08:44.362 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T20:08:44.362 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:44.362 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T20:08:44.378 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T20:08:44.434 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:08:44.434 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T20:08:44.448 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T20:08:44.500 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T20:08:44.501 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:44.501 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T20:08:44.514 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T20:08:44.570 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:08:44.571 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T20:08:44.583 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T20:08:44.637 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T20:08:44.637 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:44.638 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T20:08:44.651 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T20:08:44.706 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:08:44.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T20:08:44.718 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T20:08:44.774 INFO:teuthology.run_tasks:Running task ceph... 2024-05-09T20:08:44.864 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-09T20:08:44.864 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /var/log/ceph 2024-05-09T20:08:44.866 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /var/log/ceph 2024-05-09T20:08:44.879 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-09T20:08:44.879 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T20:08:44.923 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T20:08:44.935 INFO:tasks.ceph:Creating extra log directories... 2024-05-09T20:08:44.936 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T20:08:44.979 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T20:08:45.003 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-09T20:08:45.003 INFO:tasks.ceph:config {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-09T20:08:45.003 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi/7699763', 'branch': 'wip-rgw-d4n-long-descriptions', 'description': 'rgw:verify/{0-install accounts$/{main} clusters/fixed-2 datacache/no_datacache frontend/beast ignore-pg-availability inline-data$/{on} msgr-failures/few objectstore/bluestore-bitmap overrides proto/http rgw_pool_type/ec s3tests-branch sharding$/{default} striping$/{stripe-greater-than-chunk} supported-random-distro$/{ubuntu_latest} tasks/{bucket-check cls mp_reupload ragweed reshard s3tests-java s3tests versioning} validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7699763', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'suriarte-2024-05-09_19:24:12-rgw:verify-wip-rgw-d4n-long-descriptions-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rgw-d4n-long-descriptions'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled', '\\(OSD_SLOW_PING_TIME'], 'sha1': '04969094632d400ff0930fc029b1d19b9c187eef'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '04969094632d400ff0930fc029b1d19b9c187eef'}}, 'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}, 'compression type': 'random', 'ec-data-pool': True, 'frontend': 'beast', 'storage classes': 'LUKEWARM, FROZEN'}, 's3tests': {'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 main': 'RGW11111111111111111'}, 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rgw-d4n-long-descriptions', 'sha1': '86872c6b9201173c508ab909aaa65bdb12239dd6'}}, 'owner': 'scheduled_suriarte@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '04969094632d400ff0930fc029b1d19b9c187eef', 'sleep_before_teardown': 0, 'suite': 'rgw:verify', 'suite_branch': 'wip-rgw-d4n-long-descriptions', 'suite_path': '/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/samarahu/ceph.git', 'suite_sha1': '86872c6b9201173c508ab909aaa65bdb12239dd6', 'targets': {'smithi063.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIbTy3raqO4khV5rOqqDzh8MX5hOs+O6By/hpYAr6mpnsxb3z2nBAgcPVoX0Ayw25ENKd1f1+vX5V6ge8RfGkJM=', 'smithi120.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEOx++WTTmCNruefhchnjFlI04DbsUCT5C2bYVCLZkan++Vwy5oJe+5YugHaHyuTg9gdefdhRvRYvBmK0rpKMk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['s3cmd', 'maven'], 'rpm': ['s3cmd', 'maven']}, 'flavor': 'default', 'sha1': '04969094632d400ff0930fc029b1d19b9c187eef'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': None}}, {'tox': ['client.0']}, {'workunit': {'clients': {'client.0': ['rgw/run-bucket-check.sh']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh']}}}, {'workunit': {'clients': {'client.0': ['rgw/test_rgw_s3_mp_reupload.sh']}}}, {'ragweed': {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-reshard.sh']}}}, {'s3tests-java': {'client.0': {'force-branch': 'ceph-master', 'force-repo': 'https://github.com/ceph/java_s3tests.git'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0'}}}, {'workunit': {'clients': {'client.0': ['rgw/run-versioning.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-09_19:24:12', 'tube': 'smithi', 'user': 'suriarte', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-09T20:08:45.004 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T20:08:45.036 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T20:08:45.041 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T20:08:45.083 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T20:08:45.097 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:45.097 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T20:08:45.142 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T20:08:45.143 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-05-09T20:08:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T20:08:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T20:08:45.191 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 875 Links: 1 2024-05-09T20:08:45.191 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T20:08:45.191 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-09 20:02:40.491626109 +0000 2024-05-09T20:08:45.191 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 20:02:40.243628325 +0000 2024-05-09T20:08:45.191 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 20:02:40.243628325 +0000 2024-05-09T20:08:45.191 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-05-09T20:08:45.192 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T20:08:45.245 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-09T20:08:45.245 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-09T20:08:45.245 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.00044709 s, 1.1 MB/s 2024-05-09T20:08:45.246 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T20:08:45.296 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-05-09T20:08:45.346 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T20:08:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T20:08:45.347 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-05-09T20:08:45.347 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T20:08:45.347 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-09 20:02:40.987621679 +0000 2024-05-09T20:08:45.347 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 20:02:40.735623930 +0000 2024-05-09T20:08:45.347 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 20:02:40.735623930 +0000 2024-05-09T20:08:45.347 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-05-09T20:08:45.347 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T20:08:45.401 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-09T20:08:45.401 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-09T20:08:45.401 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000445396 s, 1.1 MB/s 2024-05-09T20:08:45.402 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T20:08:45.455 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-05-09T20:08:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T20:08:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T20:08:45.503 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-05-09T20:08:45.503 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T20:08:45.503 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-09 20:02:41.487617213 +0000 2024-05-09T20:08:45.503 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 20:02:41.207619714 +0000 2024-05-09T20:08:45.503 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 20:02:41.207619714 +0000 2024-05-09T20:08:45.503 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-05-09T20:08:45.503 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T20:08:45.557 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-09T20:08:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-09T20:08:45.558 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000394611 s, 1.3 MB/s 2024-05-09T20:08:45.559 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T20:08:45.608 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-05-09T20:08:45.654 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T20:08:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T20:08:45.655 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-05-09T20:08:45.655 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T20:08:45.655 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-05-09 20:02:41.999612639 +0000 2024-05-09T20:08:45.655 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-05-09 20:02:41.727615068 +0000 2024-05-09T20:08:45.655 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-05-09 20:02:41.727615068 +0000 2024-05-09T20:08:45.655 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: - 2024-05-09T20:08:45.655 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T20:08:45.709 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-05-09T20:08:45.709 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-05-09T20:08:45.709 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000372781 s, 1.4 MB/s 2024-05-09T20:08:45.710 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T20:08:45.760 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T20:08:45.760 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:08:45.760 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T20:08:45.767 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T20:08:45.767 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-05-09T20:08:45.815 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T20:08:45.815 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T20:08:45.815 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-05-09T20:08:45.815 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T20:08:45.815 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-05-09 20:02:37.536941912 +0000 2024-05-09T20:08:45.815 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-05-09 20:02:37.288944127 +0000 2024-05-09T20:08:45.815 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-05-09 20:02:37.288944127 +0000 2024-05-09T20:08:45.815 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2024-05-09T20:08:45.816 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T20:08:45.868 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-05-09T20:08:45.869 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-05-09T20:08:45.869 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000409649 s, 1.2 MB/s 2024-05-09T20:08:45.870 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T20:08:45.919 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-05-09T20:08:45.966 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T20:08:45.966 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T20:08:45.966 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-05-09T20:08:45.966 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T20:08:45.966 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-05-09 20:02:38.012937659 +0000 2024-05-09T20:08:45.966 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-05-09 20:02:37.756939946 +0000 2024-05-09T20:08:45.967 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-05-09 20:02:37.756939946 +0000 2024-05-09T20:08:45.967 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2024-05-09T20:08:45.967 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T20:08:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-05-09T20:08:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-05-09T20:08:46.021 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.00043189 s, 1.2 MB/s 2024-05-09T20:08:46.022 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T20:08:46.071 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-05-09T20:08:46.118 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T20:08:46.118 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T20:08:46.118 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-05-09T20:08:46.118 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T20:08:46.118 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-05-09 20:02:38.732931229 +0000 2024-05-09T20:08:46.118 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-05-09 20:02:38.248935552 +0000 2024-05-09T20:08:46.118 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-05-09 20:02:38.248935552 +0000 2024-05-09T20:08:46.118 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2024-05-09T20:08:46.119 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T20:08:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-05-09T20:08:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-05-09T20:08:46.174 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000382462 s, 1.3 MB/s 2024-05-09T20:08:46.175 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T20:08:46.223 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-05-09T20:08:46.270 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T20:08:46.270 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T20:08:46.270 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 928 Links: 1 2024-05-09T20:08:46.270 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T20:08:46.270 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-05-09 20:02:39.220926870 +0000 2024-05-09T20:08:46.271 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-05-09 20:02:38.960929191 +0000 2024-05-09T20:08:46.271 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-05-09 20:02:38.960929191 +0000 2024-05-09T20:08:46.271 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2024-05-09T20:08:46.271 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T20:08:46.324 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-05-09T20:08:46.324 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-05-09T20:08:46.325 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000352691 s, 1.5 MB/s 2024-05-09T20:08:46.326 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T20:08:46.375 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T20:08:46.375 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T20:08:46.375 INFO:tasks.ceph:Generating config... 2024-05-09T20:08:46.376 INFO:tasks.ceph:[client] debug rgw = 20 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw bucket counters cache = True 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw max chunk size = 4194304 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw obj stripe size = 6291456 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw s3 auth use sts = True 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw sts key = abcdefghijklmnop 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw torrent flag = True 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] rgw user counters cache = True 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] setgroup = ceph 2024-05-09T20:08:46.377 INFO:tasks.ceph:[client] setuser = ceph 2024-05-09T20:08:46.377 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-09T20:08:46.377 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-09T20:08:46.378 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2024-05-09T20:08:46.378 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2024-05-09T20:08:46.378 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2024-05-09T20:08:46.378 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-09T20:08:46.378 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-09T20:08:46.378 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-09T20:08:46.378 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-09T20:08:46.378 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-09T20:08:46.378 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-05-09T20:08:46.378 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-09T20:08:46.378 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-09T20:08:46.378 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-09T20:08:46.378 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-09T20:08:46.379 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-05-09T20:08:46.380 INFO:tasks.ceph:Setting up mon.a... 2024-05-09T20:08:46.380 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-09T20:08:46.414 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.keyring 2024-05-09T20:08:46.427 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-09T20:08:46.508 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T20:08:46.562 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.63'), ('mon.c', '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]'), ('mon.b', '172.21.15.120')] 2024-05-09T20:08:46.562 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790],172.21.15.120', 'mon client directed command retry': 5, 'ms inject socket failures': 5000, 'osd heartbeat grace': 40, 'osd_max_pg_log_entries': 10, 'osd_min_pg_log_entries': 10}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rgw': 20, 'rgw bucket counters cache': True, 'rgw crypt require ssl': False, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw max chunk size': 4194304, 'rgw obj stripe size': 6291456, 'rgw s3 auth use sts': True, 'rgw sts key': 'abcdefghijklmnop', 'rgw torrent flag': True, 'rgw user counters cache': True, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-05-09T20:08:46.563 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:46.563 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T20:08:46.611 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.63 --addv c '[v2:172.21.15.63:3301,v1:172.21.15.63:6790]' --add b 172.21.15.120 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:08:46.696 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:08:46.696 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:08:46.696 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:08:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool: generated fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:08:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:setting min_mon_release = quincy 2024-05-09T20:08:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 0 2024-05-09T20:08:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:08:46.697 INFO:teuthology.orchestra.run.smithi063.stdout:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:08:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:08:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:min_mon_release 17 (quincy) 2024-05-09T20:08:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:election_strategy: 1 2024-05-09T20:08:46.698 INFO:teuthology.orchestra.run.smithi063.stdout:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:08:46.701 INFO:teuthology.orchestra.run.smithi063.stdout:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:08:46.702 INFO:teuthology.orchestra.run.smithi063.stdout:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:08:46.702 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-09T20:08:46.703 DEBUG:teuthology.orchestra.run.smithi063:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T20:08:46.751 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9ed7a651-94d4-45eb-abce-d57b88f09598... 2024-05-09T20:08:46.752 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T20:08:46.795 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T20:08:46.824 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-05-09T20:08:46.824 INFO:teuthology.orchestra.run.smithi063.stdout: chdir = "" 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: auth supported = cephx 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: filestore xattr use omap = true 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: mon clock drift allowed = 1.000 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: osd crush chooseleaf type = 0 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: auth debug = true 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on old message = true 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: ms die on bug = true 2024-05-09T20:08:46.825 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: mon pg warn max object skew = 0 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default size = 2 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow primary affinity = true 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd allow pg remap = true 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on legacy crush tunables = false 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on crush straw calc version zero = false 2024-05-09T20:08:46.826 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on no sortbitwise = false 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on osd down out interval zero = false 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: mon warn on too few osds = false 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: mon_allow_pool_size_one = true 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: osd default data pool replay window = 5 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: mon allow pool delete = true 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: mon cluster log file level = debug 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: debug asserts on shutdown = true 2024-05-09T20:08:46.827 INFO:teuthology.orchestra.run.smithi063.stdout: mon health detail to clog = false 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: mon host = "172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790],172.21.15.120" 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: mon client directed command retry = 5 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: ms inject socket failures = 5000 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: osd heartbeat grace = 40 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: osd_max_pg_log_entries = 10 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: osd_min_pg_log_entries = 10 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout:[osd] 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: osd journal size = 100 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub load threshold = 5.0 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: osd scrub max interval = 600 2024-05-09T20:08:46.828 INFO:teuthology.orchestra.run.smithi063.stdout: osd mclock profile = high_recovery_ops 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd recover clone overlap = true 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd recovery max chunk = 1048576 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug shutdown = true 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug op order = true 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug verify stray on activate = true 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug trim objects = true 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd open classes on start = true 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug pg log writeout = true 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.829 INFO:teuthology.orchestra.run.smithi063.stdout: osd deep scrub update digest min age = 30 2024-05-09T20:08:46.830 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.830 INFO:teuthology.orchestra.run.smithi063.stdout: osd map max advance = 10 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: journal zero on create = true 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: filestore ondisk finisher threads = 3 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: filestore apply finisher threads = 3 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: bdev debug aio = true 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: osd debug misdirected ops = true 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: bdev async discard = True 2024-05-09T20:08:46.831 INFO:teuthology.orchestra.run.smithi063.stdout: bdev enable discard = True 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore allocator = bitmap 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore block size = 96636764160 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: bluestore fsck on mount = True 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluefs = 1/20 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: debug bluestore = 1/20 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: debug osd = 20 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: debug rocksdb = 4/10 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd full ratio = 0.9 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd nearfull ratio = 0.8 2024-05-09T20:08:46.832 INFO:teuthology.orchestra.run.smithi063.stdout: osd failsafe full ratio = 0.95 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout: osd fast shutdown = False 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout: osd objectstore = bluestore 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout: osd shutdown pgref assert = True 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr] 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout: debug mgr = 20 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-09T20:08:46.833 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-09T20:08:46.834 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-09T20:08:46.834 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-09T20:08:46.834 INFO:teuthology.orchestra.run.smithi063.stdout: mgr/telemetry/nag = false 2024-05-09T20:08:46.834 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.834 INFO:teuthology.orchestra.run.smithi063.stdout:[mon] 2024-05-09T20:08:46.834 INFO:teuthology.orchestra.run.smithi063.stdout: debug ms = 1 2024-05-09T20:08:46.835 INFO:teuthology.orchestra.run.smithi063.stdout: debug mon = 20 2024-05-09T20:08:46.835 INFO:teuthology.orchestra.run.smithi063.stdout: debug paxos = 20 2024-05-09T20:08:46.835 INFO:teuthology.orchestra.run.smithi063.stdout: debug auth = 20 2024-05-09T20:08:46.835 INFO:teuthology.orchestra.run.smithi063.stdout: mon data avail warn = 5 2024-05-09T20:08:46.835 INFO:teuthology.orchestra.run.smithi063.stdout: mon mgr mkfs grace = 240 2024-05-09T20:08:46.835 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min pgs per osd = 4 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd reporter subtree level = osd 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd prime pg temp = true 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: mon reweight min bytes per osd = 10 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T20:08:46.836 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.837 INFO:teuthology.orchestra.run.smithi063.stdout: # 1m isn't quite enough 2024-05-09T20:08:46.837 INFO:teuthology.orchestra.run.smithi063.stdout: mon_down_mkfs_grace = 2m 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: mon_warn_on_filestore_osds = false 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: mon osd crush smoke test = False 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout:[client] 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: rgw cache enabled = true 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable ops log = true 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: rgw enable usage log = true 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: debug rgw = 20 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: rgw bucket counters cache = True 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: rgw crypt require ssl = False 2024-05-09T20:08:46.838 INFO:teuthology.orchestra.run.smithi063.stdout: rgw crypt s3 kms backend = testing 2024-05-09T20:08:46.839 INFO:teuthology.orchestra.run.smithi063.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-05-09T20:08:46.839 INFO:teuthology.orchestra.run.smithi063.stdout: rgw max chunk size = 4194304 2024-05-09T20:08:46.839 INFO:teuthology.orchestra.run.smithi063.stdout: rgw obj stripe size = 6291456 2024-05-09T20:08:46.839 INFO:teuthology.orchestra.run.smithi063.stdout: rgw s3 auth use sts = True 2024-05-09T20:08:46.839 INFO:teuthology.orchestra.run.smithi063.stdout: rgw sts key = abcdefghijklmnop 2024-05-09T20:08:46.839 INFO:teuthology.orchestra.run.smithi063.stdout: rgw torrent flag = True 2024-05-09T20:08:46.840 INFO:teuthology.orchestra.run.smithi063.stdout: rgw user counters cache = True 2024-05-09T20:08:46.840 INFO:teuthology.orchestra.run.smithi063.stdout: setgroup = ceph 2024-05-09T20:08:46.840 INFO:teuthology.orchestra.run.smithi063.stdout: setuser = ceph 2024-05-09T20:08:46.840 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.a] 2024-05-09T20:08:46.840 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.c] 2024-05-09T20:08:46.840 INFO:teuthology.orchestra.run.smithi063.stdout:[mon.b] 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: chdir = "" 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: auth supported = cephx 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: filestore xattr use omap = true 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: mon clock drift allowed = 1.000 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: osd crush chooseleaf type = 0 2024-05-09T20:08:46.842 INFO:teuthology.orchestra.run.smithi120.stdout: auth debug = true 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on old message = true 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on bug = true 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: mon pg warn max object skew = 0 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default size = 2 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.843 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow primary affinity = true 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow pg remap = true 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on legacy crush tunables = false 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on crush straw calc version zero = false 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on no sortbitwise = false 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on osd down out interval zero = false 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on too few osds = false 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon_allow_pool_size_one = true 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: osd default data pool replay window = 5 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.844 INFO:teuthology.orchestra.run.smithi120.stdout: mon allow pool delete = true 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: mon cluster log file level = debug 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: debug asserts on shutdown = true 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: mon health detail to clog = false 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: mon host = "172.21.15.63,[v2:172.21.15.63:3301,v1:172.21.15.63:6790],172.21.15.120" 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: mon client directed command retry = 5 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: ms inject socket failures = 5000 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: osd heartbeat grace = 40 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: osd_max_pg_log_entries = 10 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: osd_min_pg_log_entries = 10 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.845 INFO:teuthology.orchestra.run.smithi120.stdout:[osd] 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd journal size = 100 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub load threshold = 5.0 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub max interval = 600 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd mclock profile = high_recovery_ops 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd recover clone overlap = true 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd recovery max chunk = 1048576 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug shutdown = true 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug op order = true 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug verify stray on activate = true 2024-05-09T20:08:46.846 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug trim objects = true 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: osd open classes on start = true 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug pg log writeout = true 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: osd deep scrub update digest min age = 30 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: osd map max advance = 10 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: journal zero on create = true 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: filestore ondisk finisher threads = 3 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: filestore apply finisher threads = 3 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: bdev debug aio = true 2024-05-09T20:08:46.847 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug misdirected ops = true 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: bdev async discard = True 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: bdev enable discard = True 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore allocator = bitmap 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore block size = 96636764160 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore fsck on mount = True 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluefs = 1/20 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluestore = 1/20 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: debug osd = 20 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: debug rocksdb = 4/10 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd full ratio = 0.9 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd nearfull ratio = 0.8 2024-05-09T20:08:46.848 INFO:teuthology.orchestra.run.smithi120.stdout: osd failsafe full ratio = 0.95 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: osd fast shutdown = False 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: osd objectstore = bluestore 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: osd shutdown pgref assert = True 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr] 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: debug mgr = 20 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: mgr/telemetry/nag = false 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout:[mon] 2024-05-09T20:08:46.849 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: debug paxos = 20 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: mon data avail warn = 5 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: mon mgr mkfs grace = 240 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd reporter subtree level = osd 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd prime pg temp = true 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T20:08:46.850 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: # 1m isn't quite enough 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: mon_down_mkfs_grace = 2m 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_filestore_osds = false 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd crush smoke test = False 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout:[client] 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: rgw cache enabled = true 2024-05-09T20:08:46.851 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable ops log = true 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable usage log = true 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: debug rgw = 20 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw bucket counters cache = True 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw crypt require ssl = False 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw crypt s3 kms backend = testing 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw max chunk size = 4194304 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw obj stripe size = 6291456 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw s3 auth use sts = True 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw sts key = abcdefghijklmnop 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw torrent flag = True 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: rgw user counters cache = True 2024-05-09T20:08:46.852 INFO:teuthology.orchestra.run.smithi120.stdout: setgroup = ceph 2024-05-09T20:08:46.853 INFO:teuthology.orchestra.run.smithi120.stdout: setuser = ceph 2024-05-09T20:08:46.853 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.a] 2024-05-09T20:08:46.853 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.c] 2024-05-09T20:08:46.853 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.b] 2024-05-09T20:08:46.853 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-09T20:08:46.853 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-09T20:08:46.942 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-09T20:08:46.942 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:46.942 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-09T20:08:46.947 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:46.947 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-09T20:08:46.994 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T20:08:46.994 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:46.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T20:08:46.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T20:08:47.059 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:08:47.059 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:08:47.106 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi120.front.sepia.ceph.com 2024-05-09T20:08:47.107 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:08:47.107 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T20:08:47.107 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T20:08:47.129 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:08:47.129 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:08:47.178 INFO:tasks.ceph:Setting up mon nodes... 2024-05-09T20:08:47.178 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-09T20:08:47.178 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T20:08:47.223 INFO:teuthology.orchestra.run.smithi063.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T20:08:47.226 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T20:08:47.271 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T20:08:47.279 INFO:tasks.ceph:Setting up mds nodes... 2024-05-09T20:08:47.279 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-09T20:08:47.279 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T20:08:47.318 INFO:teuthology.orchestra.run.smithi063.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-09T20:08:47.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-09T20:08:47.362 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-09T20:08:47.372 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-09T20:08:47.372 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi063.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T20:08:47.372 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-09T20:08:47.384 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T20:08:47.384 INFO:tasks.ceph:role: osd.0 2024-05-09T20:08:47.384 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T20:08:47.384 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-09T20:08:47.447 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T20:08:47.448 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T20:08:47.900 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-09T20:08:47.902 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-09T20:08:47.902 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-09T20:08:47.990 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-09T20:08:48.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:08:48.043 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-05-09T20:08:48.044 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-09T20:08:48.096 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T20:08:48.096 INFO:tasks.ceph:role: osd.1 2024-05-09T20:08:48.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T20:08:48.097 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T20:08:48.159 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T20:08:48.642 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-09T20:08:48.645 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-09T20:08:48.645 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-09T20:08:48.778 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-09T20:08:48.789 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-05-09T20:08:48.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:08:48.791 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-09T20:08:48.846 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T20:08:48.846 INFO:tasks.ceph:role: osd.2 2024-05-09T20:08:48.846 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T20:08:48.846 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T20:08:48.911 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T20:08:49.385 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-09T20:08:49.388 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-09T20:08:49.388 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-09T20:08:49.526 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-09T20:08:49.538 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-05-09T20:08:49.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:08:49.540 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-09T20:08:49.594 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T20:08:49.595 INFO:tasks.ceph:role: osd.3 2024-05-09T20:08:49.595 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T20:08:49.595 DEBUG:teuthology.orchestra.run.smithi063:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T20:08:49.658 INFO:teuthology.orchestra.run.smithi063.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T20:08:49.658 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T20:08:49.658 INFO:teuthology.orchestra.run.smithi063.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T20:08:49.659 INFO:teuthology.orchestra.run.smithi063.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-09T20:08:49.659 INFO:teuthology.orchestra.run.smithi063.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T20:08:49.659 INFO:teuthology.orchestra.run.smithi063.stdout: = sunit=0 swidth=0 blks 2024-05-09T20:08:49.659 INFO:teuthology.orchestra.run.smithi063.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T20:08:49.659 INFO:teuthology.orchestra.run.smithi063.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-09T20:08:49.659 INFO:teuthology.orchestra.run.smithi063.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T20:08:49.659 INFO:teuthology.orchestra.run.smithi063.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T20:08:50.085 INFO:teuthology.orchestra.run.smithi063.stdout:Discarding blocks...Done. 2024-05-09T20:08:50.088 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi063.front.sepia.ceph.com -o noatime 2024-05-09T20:08:50.088 DEBUG:teuthology.orchestra.run.smithi063:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-09T20:08:50.187 DEBUG:teuthology.orchestra.run.smithi063:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-09T20:08:50.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:08:50.197 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: /sbin/restorecon: command not found 2024-05-09T20:08:50.197 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:08:50.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:50.308+0000 7f026f8817c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-09T20:08:50.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:50.308+0000 7f026f8817c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-09T20:08:50.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:50.308+0000 7f026f8817c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T20:08:50.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:50.308+0000 7f026f8817c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T20:08:50.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:50.308+0000 7f026f8817c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T20:08:50.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:50.316+0000 7f026f8817c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-09T20:08:53.278 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T20:08:53.292 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:08:53.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:53.379+0000 7fd67d6f87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-09T20:08:53.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:53.379+0000 7fd67d6f87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-09T20:08:53.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:53.379+0000 7fd67d6f87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T20:08:53.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:53.379+0000 7fd67d6f87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T20:08:53.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:53.379+0000 7fd67d6f87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T20:08:53.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:53.379+0000 7fd67d6f87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-09T20:08:56.377 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T20:08:56.434 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:08:56.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:56.523+0000 7f52b41417c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-09T20:08:56.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:56.523+0000 7f52b41417c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-09T20:08:56.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:56.523+0000 7f52b41417c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T20:08:56.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:56.523+0000 7f52b41417c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T20:08:56.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:56.523+0000 7f52b41417c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T20:08:56.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:56.523+0000 7f52b41417c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-09T20:08:59.486 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T20:08:59.546 DEBUG:teuthology.orchestra.run.smithi063:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:08:59.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:59.639+0000 7fc0b360f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-09T20:08:59.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:59.639+0000 7fc0b360f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-09T20:08:59.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:59.639+0000 7fc0b360f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T20:08:59.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:59.639+0000 7fc0b360f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T20:08:59.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:59.639+0000 7fc0b360f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T20:08:59.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:08:59.639+0000 7fc0b360f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-09T20:09:02.610 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T20:09:02.670 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-09T20:09:02.685 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T20:09:02.685 INFO:tasks.ceph:role: osd.4 2024-05-09T20:09:02.685 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com 2024-05-09T20:09:02.685 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T20:09:02.750 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T20:09:02.750 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T20:09:02.750 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T20:09:02.750 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-09T20:09:02.750 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T20:09:02.751 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-05-09T20:09:02.751 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T20:09:02.751 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-09T20:09:02.751 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T20:09:02.751 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T20:09:03.213 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-05-09T20:09:03.215 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-05-09T20:09:03.216 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-09T20:09:03.340 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-09T20:09:03.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:09:03.351 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: /sbin/restorecon: command not found 2024-05-09T20:09:03.351 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-09T20:09:03.404 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T20:09:03.404 INFO:tasks.ceph:role: osd.5 2024-05-09T20:09:03.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com 2024-05-09T20:09:03.405 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-09T20:09:03.466 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T20:09:03.467 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T20:09:03.906 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-05-09T20:09:03.907 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-05-09T20:09:03.908 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-09T20:09:03.970 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-09T20:09:04.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:09:04.024 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: /sbin/restorecon: command not found 2024-05-09T20:09:04.024 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-09T20:09:04.077 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T20:09:04.078 INFO:tasks.ceph:role: osd.6 2024-05-09T20:09:04.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com 2024-05-09T20:09:04.078 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T20:09:04.142 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T20:09:04.142 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T20:09:04.142 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T20:09:04.142 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-09T20:09:04.142 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T20:09:04.142 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-05-09T20:09:04.143 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T20:09:04.143 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-09T20:09:04.143 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T20:09:04.143 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T20:09:04.608 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-05-09T20:09:04.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-05-09T20:09:04.610 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-09T20:09:04.706 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-09T20:09:04.717 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: /sbin/restorecon: command not found 2024-05-09T20:09:04.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:09:04.718 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-09T20:09:04.774 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T20:09:04.774 INFO:tasks.ceph:role: osd.7 2024-05-09T20:09:04.774 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com 2024-05-09T20:09:04.774 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T20:09:04.838 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T20:09:04.838 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T20:09:04.838 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T20:09:04.838 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-05-09T20:09:04.838 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T20:09:04.838 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-05-09T20:09:04.838 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T20:09:04.838 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-05-09T20:09:04.839 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T20:09:04.839 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T20:09:05.297 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-05-09T20:09:05.299 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-05-09T20:09:05.299 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-09T20:09:05.430 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-09T20:09:05.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:09:05.441 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: /sbin/restorecon: command not found 2024-05-09T20:09:05.441 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:09:05.632 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:05.637+0000 7f4e2e1297c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-09T20:09:05.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:05.641+0000 7f4e2e1297c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-09T20:09:05.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:05.641+0000 7f4e2e1297c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T20:09:05.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:05.641+0000 7f4e2e1297c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T20:09:05.637 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:05.641+0000 7f4e2e1297c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T20:09:05.644 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:05.649+0000 7f4e2e1297c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-09T20:09:08.657 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T20:09:08.671 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:09:08.751 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:08.757+0000 7f4b2f5577c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-09T20:09:08.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:08.757+0000 7f4b2f5577c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-09T20:09:08.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:08.757+0000 7f4b2f5577c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T20:09:08.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:08.757+0000 7f4b2f5577c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T20:09:08.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:08.757+0000 7f4b2f5577c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T20:09:08.753 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:08.757+0000 7f4b2f5577c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-09T20:09:11.718 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T20:09:11.774 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:09:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:11.861+0000 7f7a9c1f67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-09T20:09:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:11.861+0000 7f7a9c1f67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-09T20:09:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:11.861+0000 7f7a9c1f67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T20:09:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:11.861+0000 7f7a9c1f67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T20:09:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:11.861+0000 7f7a9c1f67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T20:09:11.859 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:11.865+0000 7f7a9c1f67c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-09T20:09:14.825 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-09T20:09:14.882 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:09:14.963 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:14.969+0000 7f96529957c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-09T20:09:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:14.969+0000 7f96529957c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-09T20:09:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:14.969+0000 7f96529957c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T20:09:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:14.969+0000 7f96529957c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T20:09:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:14.969+0000 7f96529957c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T20:09:14.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-05-09T20:09:14.969+0000 7f96529957c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-09T20:09:17.934 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-09T20:09:17.990 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-09T20:09:17.991 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:17.991 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-09T20:09:18.006 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:18.006 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-09T20:09:18.061 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:18.061 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-09T20:09:18.117 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:18.117 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-09T20:09:18.174 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:18.174 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-09T20:09:18.230 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:18.230 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-09T20:09:18.243 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:18.243 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-09T20:09:18.297 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:18.297 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-09T20:09:18.353 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:18.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-09T20:09:18.408 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:18.409 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-09T20:09:18.465 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:18.465 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-09T20:09:18.471 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:18.471 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-09T20:09:18.513 INFO:tasks.ceph:Adding keys to all mons... 2024-05-09T20:09:18.514 DEBUG:teuthology.orchestra.run.smithi063:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T20:09:18.516 DEBUG:teuthology.orchestra.run.smithi120:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.y] 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBPLT1mPbrJDRAASmsTrc/dOp26D8wOd4Fgyg== 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.0] 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBSLT1mKoCoEhAA9ohKfQLSHTEmb62Ci4XB/g== 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.1] 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBVLT1m4abFFhAASVPbZk0fBdT43TxMycHBxg== 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.2] 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBYLT1m4LVdHxAAuRkbro1TywWJ0wzgcePSQw== 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.3] 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBbLT1miM5jJhAAdVTQz4aChgfhFTfnj0W4yA== 2024-05-09T20:09:18.560 INFO:teuthology.orchestra.run.smithi063.stdout:[mgr.x] 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBPLT1mr0q3EBAAovlA4rRkgBTj/pu78J3oZQ== 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.4] 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBhLT1meQ5IJhAAIMdxrElhpLtxz9ZcaM52Ew== 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.5] 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBkLT1mdgBvLRAAGDxqelStfQgh/4nvRoD3MQ== 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.6] 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBnLT1m8xytMxAAd1BfpwN78k84kaOUJHWNkQ== 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout:[osd.7] 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBqLT1mew4KOhAA+07Qdq3lb5/OuVFcXFZyUw== 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBPLT1mj3BpExAApXu8fyiaTv7I6vmGVd+aLQ== 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout:[client.1] 2024-05-09T20:09:18.561 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBPLT1mxjwvFhAAUHJ9y0oJ0H1XOu+BeZEogA== 2024-05-09T20:09:18.567 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.y] 2024-05-09T20:09:18.567 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBPLT1mPbrJDRAASmsTrc/dOp26D8wOd4Fgyg== 2024-05-09T20:09:18.567 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.0] 2024-05-09T20:09:18.567 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBSLT1mKoCoEhAA9ohKfQLSHTEmb62Ci4XB/g== 2024-05-09T20:09:18.567 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.1] 2024-05-09T20:09:18.567 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBVLT1m4abFFhAASVPbZk0fBdT43TxMycHBxg== 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.2] 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBYLT1m4LVdHxAAuRkbro1TywWJ0wzgcePSQw== 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.3] 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBbLT1miM5jJhAAdVTQz4aChgfhFTfnj0W4yA== 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.x] 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBPLT1mr0q3EBAAovlA4rRkgBTj/pu78J3oZQ== 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.4] 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBhLT1meQ5IJhAAIMdxrElhpLtxz9ZcaM52Ew== 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.5] 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBkLT1mdgBvLRAAGDxqelStfQgh/4nvRoD3MQ== 2024-05-09T20:09:18.568 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.6] 2024-05-09T20:09:18.569 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBnLT1m8xytMxAAd1BfpwN78k84kaOUJHWNkQ== 2024-05-09T20:09:18.569 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.7] 2024-05-09T20:09:18.569 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBqLT1mew4KOhAA+07Qdq3lb5/OuVFcXFZyUw== 2024-05-09T20:09:18.569 INFO:teuthology.orchestra.run.smithi120.stdout:[client.0] 2024-05-09T20:09:18.569 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBPLT1mj3BpExAApXu8fyiaTv7I6vmGVd+aLQ== 2024-05-09T20:09:18.569 INFO:teuthology.orchestra.run.smithi120.stdout:[client.1] 2024-05-09T20:09:18.569 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBPLT1mxjwvFhAAUHJ9y0oJ0H1XOu+BeZEogA== 2024-05-09T20:09:18.570 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T20:09:18.607 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T20:09:18.657 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:18.660 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:18.700 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:18.743 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:18.787 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:18.831 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:18.872 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:18.915 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:18.960 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T20:09:18.999 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T20:09:19.046 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:19.048 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:19.090 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:19.092 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:19.168 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:19.175 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:19.253 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:19.259 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T20:09:19.342 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T20:09:19.344 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T20:09:19.391 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T20:09:19.435 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T20:09:19.479 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-09T20:09:19.479 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-09T20:09:19.532 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T20:09:19.620 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-09T20:09:19.631 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-09T20:09:19.683 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T20:09:19.773 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-09T20:09:19.785 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-09T20:09:19.799 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T20:09:19.893 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-09T20:09:19.904 DEBUG:teuthology.orchestra.run.smithi063:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:09:19.906 DEBUG:teuthology.orchestra.run.smithi120:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T20:09:19.949 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-09T20:09:19.949 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T20:09:19.949 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T20:09:19.955 INFO:tasks.ceph.mon.a:Started 2024-05-09T20:09:19.955 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T20:09:19.955 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T20:09:19.958 INFO:tasks.ceph.mon.c:Started 2024-05-09T20:09:19.958 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T20:09:19.958 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T20:09:19.994 INFO:tasks.ceph.mon.b:Started 2024-05-09T20:09:19.994 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-09T20:09:19.995 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-09T20:09:19.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-09T20:09:19.998 INFO:tasks.ceph.mgr.y:Started 2024-05-09T20:09:19.998 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-09T20:09:19.998 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-09T20:09:20.000 INFO:tasks.ceph.mgr.x:Started 2024-05-09T20:09:20.000 INFO:tasks.ceph:Setting crush tunables to default 2024-05-09T20:09:20.000 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd crush tunables default 2024-05-09T20:09:25.650 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:25.655+0000 7f8622b80180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T20:09:25.742 INFO:teuthology.orchestra.run.smithi063.stderr:adjusted tunables profile to default 2024-05-09T20:09:25.762 INFO:tasks.ceph:check_enable_crimson: False 2024-05-09T20:09:25.762 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-09T20:09:25.762 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:25.762 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-09T20:09:25.774 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:25.774 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-09T20:09:25.829 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:25.829 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-09T20:09:25.861 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:25.865+0000 7f008ebe0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T20:09:25.881 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:09:25.881 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-09T20:09:25.934 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:25.934 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-09T20:09:25.949 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:25.949 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-09T20:09:26.005 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:26.005 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-09T20:09:26.060 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-05-09T20:09:26.060 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-09T20:09:26.112 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 4e7e57ca-2626-45fa-9a52-a35034d753ab 0 2024-05-09T20:09:26.143 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:26.147+0000 7f8622b80180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T20:09:26.319 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:26.325+0000 7f008ebe0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T20:09:26.358 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:26.363+0000 7f8622b80180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T20:09:26.358 INFO:teuthology.orchestra.run.smithi120.stdout:0 2024-05-09T20:09:26.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 442dbd50-2f65-4502-90fa-f126029d0c95 1 2024-05-09T20:09:26.530 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:26.537+0000 7f008ebe0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T20:09:26.563 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-05-09T20:09:26.580 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 5fed96b2-d11d-4f7b-9b07-9b43d41614d5 2 2024-05-09T20:09:26.767 INFO:teuthology.orchestra.run.smithi120.stdout:2 2024-05-09T20:09:26.784 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new a5c1635f-89ab-42aa-ad1e-f14da99b91b1 3 2024-05-09T20:09:26.928 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:26.931+0000 7f8622b80180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T20:09:26.977 INFO:teuthology.orchestra.run.smithi120.stdout:3 2024-05-09T20:09:26.985 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:26.991+0000 7f8622b80180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T20:09:26.994 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 62a4529d-709e-4eb6-951c-1cf3ab63d84a 4 2024-05-09T20:09:27.040 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:27.043+0000 7f8622b80180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T20:09:27.119 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.125+0000 7f008ebe0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T20:09:27.121 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:27.123+0000 7f8622b80180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T20:09:27.174 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.181+0000 7f008ebe0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T20:09:27.175 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:27.179+0000 7f8622b80180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T20:09:27.184 INFO:teuthology.orchestra.run.smithi120.stdout:4 2024-05-09T20:09:27.200 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new cb37c5ed-ec5c-4ac2-af3f-9e16c200807e 5 2024-05-09T20:09:27.231 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.237+0000 7f008ebe0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T20:09:27.260 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:27.263+0000 7f8622b80180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T20:09:27.311 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.317+0000 7f008ebe0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T20:09:27.316 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:27.319+0000 7f8622b80180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T20:09:27.366 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.373+0000 7f008ebe0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T20:09:27.393 INFO:teuthology.orchestra.run.smithi120.stdout:5 2024-05-09T20:09:27.410 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new d31c5357-5fcf-4da6-9e72-2f0f7f036acb 6 2024-05-09T20:09:27.447 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.452+0000 7f008ebe0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T20:09:27.482 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:27.487+0000 7f8622b80180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T20:09:27.502 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.508+0000 7f008ebe0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T20:09:27.549 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:27.551+0000 7f8622b80180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T20:09:27.651 INFO:teuthology.orchestra.run.smithi120.stdout:6 2024-05-09T20:09:27.657 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.664+0000 7f008ebe0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T20:09:27.667 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 5f957a83-94b2-4302-a0e3-f77757f341e2 7 2024-05-09T20:09:27.722 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:27.728+0000 7f008ebe0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T20:09:27.858 INFO:teuthology.orchestra.run.smithi120.stdout:7 2024-05-09T20:09:27.874 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-09T20:09:27.874 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-09T20:09:27.877 INFO:tasks.ceph.osd.0:Started 2024-05-09T20:09:27.877 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-09T20:09:27.877 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-09T20:09:27.879 INFO:tasks.ceph.osd.1:Started 2024-05-09T20:09:27.879 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T20:09:27.879 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T20:09:27.882 INFO:tasks.ceph.osd.2:Started 2024-05-09T20:09:27.882 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-09T20:09:27.882 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-09T20:09:27.886 INFO:tasks.ceph.osd.3:Started 2024-05-09T20:09:27.886 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-09T20:09:27.886 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-09T20:09:27.888 INFO:tasks.ceph.osd.4:Started 2024-05-09T20:09:27.889 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-09T20:09:27.889 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-09T20:09:27.890 INFO:tasks.ceph.osd.5:Started 2024-05-09T20:09:27.891 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-09T20:09:27.891 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-09T20:09:27.892 INFO:tasks.ceph.osd.6:Started 2024-05-09T20:09:27.892 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-09T20:09:27.893 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-09T20:09:27.894 INFO:tasks.ceph.osd.7:Started 2024-05-09T20:09:27.895 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T20:09:28.129 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:09:28.129 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":10,"fsid":"9ed7a651-94d4-45eb-abce-d57b88f09598","created":"2024-05-09T20:09:25.359123+0000","modified":"2024-05-09T20:09:27.861516+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T20:09:27.861516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4e7e57ca-2626-45fa-9a52-a35034d753ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"442dbd50-2f65-4502-90fa-f126029d0c95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5fed96b2-d11d-4f7b-9b07-9b43d41614d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a5c1635f-89ab-42aa-ad1e-f14da99b91b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62a4529d-709e-4eb6-951c-1cf3ab63d84a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cb37c5ed-ec5c-4ac2-af3f-9e16c200807e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d31c5357-5fcf-4da6-9e72-2f0f7f036acb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5f957a83-94b2-4302-a0e3-f77757f341e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T20:09:28.146 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-09T20:09:28.147 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-09T20:09:28.214 INFO:tasks.ceph.mgr.y.smithi063.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-09T20:09:28.214 INFO:tasks.ceph.mgr.y.smithi063.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-09T20:09:28.214 INFO:tasks.ceph.mgr.y.smithi063.stderr: from numpy import show_config as show_numpy_config 2024-05-09T20:09:28.284 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:28.287+0000 7f8622b80180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T20:09:28.443 INFO:tasks.ceph.mgr.x.smithi120.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-05-09T20:09:28.443 INFO:tasks.ceph.mgr.x.smithi120.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-05-09T20:09:28.443 INFO:tasks.ceph.mgr.x.smithi120.stderr: from numpy import show_config as show_numpy_config 2024-05-09T20:09:28.471 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:28.475+0000 7f8622b80180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T20:09:28.511 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:28.516+0000 7f008ebe0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T20:09:28.512 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-05-09T20:09:28.516+0000 7efeba2327c0 -1 Falling back to public interface 2024-05-09T20:09:28.512 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-05-09T20:09:28.516+0000 7f701894c7c0 -1 Falling back to public interface 2024-05-09T20:09:28.512 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-05-09T20:09:28.516+0000 7fb33aab57c0 -1 Falling back to public interface 2024-05-09T20:09:28.513 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-05-09T20:09:28.516+0000 7efc5c8457c0 -1 Falling back to public interface 2024-05-09T20:09:28.513 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-05-09T20:09:28.516+0000 7f701790c640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-05-09T20:09:28.535 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-05-09T20:09:28.539+0000 7f71151e57c0 -1 Falling back to public interface 2024-05-09T20:09:28.536 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-05-09T20:09:28.539+0000 7fbdf2ee07c0 -1 Falling back to public interface 2024-05-09T20:09:28.550 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-05-09T20:09:28.555+0000 7ffb4423d7c0 -1 Falling back to public interface 2024-05-09T20:09:28.558 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-05-09T20:09:28.563+0000 7fe7b7ca27c0 -1 Falling back to public interface 2024-05-09T20:09:28.656 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:28.659+0000 7f8622b80180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T20:09:28.689 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:28.692+0000 7f008ebe0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T20:09:28.727 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:28.731+0000 7f8622b80180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T20:09:28.844 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:28.848+0000 7f008ebe0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T20:09:28.877 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:28.879+0000 7f8622b80180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T20:09:28.908 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:28.912+0000 7f008ebe0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T20:09:28.952 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T20:09:29.041 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:29.044+0000 7f008ebe0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T20:09:29.138 INFO:teuthology.misc.health.smithi063.stdout: 2024-05-09T20:09:29.138 INFO:teuthology.misc.health.smithi063.stdout:{"epoch":10,"fsid":"9ed7a651-94d4-45eb-abce-d57b88f09598","created":"2024-05-09T20:09:25.359123+0000","modified":"2024-05-09T20:09:27.861516+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T20:09:27.861516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4e7e57ca-2626-45fa-9a52-a35034d753ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"442dbd50-2f65-4502-90fa-f126029d0c95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5fed96b2-d11d-4f7b-9b07-9b43d41614d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a5c1635f-89ab-42aa-ad1e-f14da99b91b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62a4529d-709e-4eb6-951c-1cf3ab63d84a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cb37c5ed-ec5c-4ac2-af3f-9e16c200807e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d31c5357-5fcf-4da6-9e72-2f0f7f036acb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5f957a83-94b2-4302-a0e3-f77757f341e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T20:09:29.153 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T20:09:29.538 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:29.543+0000 7f8622b80180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T20:09:29.616 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:29.619+0000 7f8622b80180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T20:09:29.653 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:29.656+0000 7f008ebe0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T20:09:29.733 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:29.732+0000 7f008ebe0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T20:09:29.807 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:29.811+0000 7f8622b80180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T20:09:29.883 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:29.887+0000 7f8622b80180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T20:09:29.909 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:29.912+0000 7f008ebe0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T20:09:29.981 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:29.984+0000 7f008ebe0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T20:09:30.190 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:30.195+0000 7f8622b80180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T20:09:30.254 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:30.259+0000 7f8622b80180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T20:09:30.270 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:30.272+0000 7f008ebe0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T20:09:30.331 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-05-09T20:09:30.336+0000 7f008ebe0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T20:09:31.296 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:31.299+0000 7f85c97c8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T20:09:31.297 INFO:tasks.ceph.mgr.y.smithi063.stderr:2024-05-09T20:09:31.299+0000 7f85c97c8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T20:09:32.318 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-05-09T20:09:32.324+0000 7efeba2327c0 -1 osd.4 0 log_to_monitors true 2024-05-09T20:09:32.326 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-05-09T20:09:32.332+0000 7efc5c8457c0 -1 osd.7 0 log_to_monitors true 2024-05-09T20:09:32.372 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-05-09T20:09:32.376+0000 7fb33aab57c0 -1 osd.6 0 log_to_monitors true 2024-05-09T20:09:32.412 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-05-09T20:09:32.415+0000 7f71151e57c0 -1 osd.0 0 log_to_monitors true 2024-05-09T20:09:32.460 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-05-09T20:09:32.464+0000 7f701894c7c0 -1 osd.5 0 log_to_monitors true 2024-05-09T20:09:32.491 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-05-09T20:09:32.495+0000 7fe7b7ca27c0 -1 osd.3 0 log_to_monitors true 2024-05-09T20:09:32.500 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-05-09T20:09:32.503+0000 7fbdf2ee07c0 -1 osd.1 0 log_to_monitors true 2024-05-09T20:09:32.564 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-05-09T20:09:32.567+0000 7ffb4423d7c0 -1 osd.2 0 log_to_monitors true 2024-05-09T20:09:35.960 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T20:09:36.397 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-05-09T20:09:36.400+0000 7efc587ee640 -1 osd.7 0 waiting for initial osdmap 2024-05-09T20:09:36.418 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-05-09T20:09:36.423+0000 7ffb401e6640 -1 osd.2 0 waiting for initial osdmap 2024-05-09T20:09:36.430 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-05-09T20:09:36.436+0000 7efc50622640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T20:09:36.432 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-05-09T20:09:36.436+0000 7f70148f5640 -1 osd.5 0 waiting for initial osdmap 2024-05-09T20:09:36.453 INFO:tasks.ceph.osd.2.smithi063.stderr:2024-05-09T20:09:36.455+0000 7ffb3801a640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T20:09:36.454 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-05-09T20:09:36.456+0000 7fb337270640 -1 osd.6 0 waiting for initial osdmap 2024-05-09T20:09:36.461 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-05-09T20:09:36.464+0000 7efeb69ed640 -1 osd.4 0 waiting for initial osdmap 2024-05-09T20:09:36.471 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-05-09T20:09:36.476+0000 7f700c729640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T20:09:36.476 INFO:teuthology.misc.health.smithi063.stdout: 2024-05-09T20:09:36.476 INFO:teuthology.misc.health.smithi063.stdout:{"epoch":12,"fsid":"9ed7a651-94d4-45eb-abce-d57b88f09598","created":"2024-05-09T20:09:25.359123+0000","modified":"2024-05-09T20:09:34.309636+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T20:09:27.861516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"4e7e57ca-2626-45fa-9a52-a35034d753ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"442dbd50-2f65-4502-90fa-f126029d0c95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5fed96b2-d11d-4f7b-9b07-9b43d41614d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a5c1635f-89ab-42aa-ad1e-f14da99b91b1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"62a4529d-709e-4eb6-951c-1cf3ab63d84a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cb37c5ed-ec5c-4ac2-af3f-9e16c200807e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d31c5357-5fcf-4da6-9e72-2f0f7f036acb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5f957a83-94b2-4302-a0e3-f77757f341e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T20:09:36.490 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-05-09T20:09:36.496+0000 7fb32e892640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T20:09:36.503 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T20:09:36.510 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-05-09T20:09:36.516+0000 7efeae00f640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T20:09:36.511 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-05-09T20:09:36.515+0000 7fbdeee89640 -1 osd.1 0 waiting for initial osdmap 2024-05-09T20:09:36.528 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-05-09T20:09:36.531+0000 7fe7b445d640 -1 osd.3 0 waiting for initial osdmap 2024-05-09T20:09:36.534 INFO:tasks.ceph.osd.1.smithi063.stderr:2024-05-09T20:09:36.539+0000 7fbde6cbd640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T20:09:36.562 INFO:tasks.ceph.osd.3.smithi063.stderr:2024-05-09T20:09:36.567+0000 7fe7aba7f640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T20:09:36.572 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-05-09T20:09:36.575+0000 7f711118e640 -1 osd.0 0 waiting for initial osdmap 2024-05-09T20:09:36.606 INFO:tasks.ceph.osd.0.smithi063.stderr:2024-05-09T20:09:36.611+0000 7f7108fc2640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T20:09:43.311 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T20:09:43.619 INFO:teuthology.misc.health.smithi063.stdout: 2024-05-09T20:09:43.619 INFO:teuthology.misc.health.smithi063.stdout:{"epoch":16,"fsid":"9ed7a651-94d4-45eb-abce-d57b88f09598","created":"2024-05-09T20:09:25.359123+0000","modified":"2024-05-09T20:09:40.496571+0000","last_up_change":"2024-05-09T20:09:37.441738+0000","last_in_change":"2024-05-09T20:09:27.861516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T20:09:38.336167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4e7e57ca-2626-45fa-9a52-a35034d753ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6801","nonce":3125497649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6806","nonce":3125497649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6813","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6815","nonce":3125497649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6809","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6811","nonce":3125497649}]},"public_addr":"172.21.15.63:6801/3125497649","cluster_addr":"172.21.15.63:6806/3125497649","heartbeat_back_addr":"172.21.15.63:6815/3125497649","heartbeat_front_addr":"172.21.15.63:6811/3125497649","state":["exists","up"]},{"osd":1,"uuid":"442dbd50-2f65-4502-90fa-f126029d0c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6803","nonce":3768131182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6805","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6807","nonce":3768131182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6814","nonce":3768131182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6810","nonce":3768131182}]},"public_addr":"172.21.15.63:6803/3768131182","cluster_addr":"172.21.15.63:6807/3768131182","heartbeat_back_addr":"172.21.15.63:6814/3768131182","heartbeat_front_addr":"172.21.15.63:6810/3768131182","state":["exists","up"]},{"osd":2,"uuid":"5fed96b2-d11d-4f7b-9b07-9b43d41614d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6817","nonce":1087089498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6819","nonce":1087089498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6823","nonce":1087089498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6821","nonce":1087089498}]},"public_addr":"172.21.15.63:6817/1087089498","cluster_addr":"172.21.15.63:6819/1087089498","heartbeat_back_addr":"172.21.15.63:6823/1087089498","heartbeat_front_addr":"172.21.15.63:6821/1087089498","state":["exists","up"]},{"osd":3,"uuid":"a5c1635f-89ab-42aa-ad1e-f14da99b91b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6825","nonce":1568250839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6827","nonce":1568250839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6831","nonce":1568250839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6829","nonce":1568250839}]},"public_addr":"172.21.15.63:6825/1568250839","cluster_addr":"172.21.15.63:6827/1568250839","heartbeat_back_addr":"172.21.15.63:6831/1568250839","heartbeat_front_addr":"172.21.15.63:6829/1568250839","state":["exists","up"]},{"osd":4,"uuid":"62a4529d-709e-4eb6-951c-1cf3ab63d84a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6801","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6803","nonce":982132549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6810","nonce":982132549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6828","nonce":982132549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6818","nonce":982132549}]},"public_addr":"172.21.15.120:6803/982132549","cluster_addr":"172.21.15.120:6810/982132549","heartbeat_back_addr":"172.21.15.120:6828/982132549","heartbeat_front_addr":"172.21.15.120:6818/982132549","state":["exists","up"]},{"osd":5,"uuid":"cb37c5ed-ec5c-4ac2-af3f-9e16c200807e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6809","nonce":1971324730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6813","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6816","nonce":1971324730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6830","nonce":1971324730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6819","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6822","nonce":1971324730}]},"public_addr":"172.21.15.120:6809/1971324730","cluster_addr":"172.21.15.120:6816/1971324730","heartbeat_back_addr":"172.21.15.120:6830/1971324730","heartbeat_front_addr":"172.21.15.120:6822/1971324730","state":["exists","up"]},{"osd":6,"uuid":"d31c5357-5fcf-4da6-9e72-2f0f7f036acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6807","nonce":610448732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6817","nonce":610448732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6829","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6831","nonce":610448732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6821","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6825","nonce":610448732}]},"public_addr":"172.21.15.120:6807/610448732","cluster_addr":"172.21.15.120:6817/610448732","heartbeat_back_addr":"172.21.15.120:6831/610448732","heartbeat_front_addr":"172.21.15.120:6825/610448732","state":["exists","up"]},{"osd":7,"uuid":"5f957a83-94b2-4302-a0e3-f77757f341e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6805","nonce":3964176807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6811","nonce":3964176807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6827","nonce":3964176807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6820","nonce":3964176807}]},"public_addr":"172.21.15.120:6805/3964176807","cluster_addr":"172.21.15.120:6811/3964176807","heartbeat_back_addr":"172.21.15.120:6827/3964176807","heartbeat_front_addr":"172.21.15.120:6820/3964176807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.456405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.556324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.583822+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.539726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.353298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.493080+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.420249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.344365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T20:09:43.637 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-09T20:09:43.637 INFO:tasks.ceph:Creating RBD pool 2024-05-09T20:09:43.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-09T20:09:44.680 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'rbd' created 2024-05-09T20:09:44.703 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-09T20:09:45.698 INFO:teuthology.orchestra.run.smithi063.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-09T20:09:45.726 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-09T20:09:45.726 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T20:09:45.726 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-09T20:09:45.727 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-09T20:09:45.727 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-09T20:09:46.085 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:09:46.103 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6832","nonce":2991883022},{"type":"v1","addr":"172.21.15.63:6833","nonce":2991883022}]},"active_addr":"172.21.15.63:6833/2991883022","active_change":"2024-05-09T20:09:30.283244+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1816345826}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1721337523}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":317189161}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":3242386177}]}]} 2024-05-09T20:09:46.109 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-09T20:09:46.109 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T20:09:46.110 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T20:09:46.407 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:09:46.407 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":18,"fsid":"9ed7a651-94d4-45eb-abce-d57b88f09598","created":"2024-05-09T20:09:25.359123+0000","modified":"2024-05-09T20:09:45.688962+0000","last_up_change":"2024-05-09T20:09:37.441738+0000","last_in_change":"2024-05-09T20:09:27.861516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T20:09:38.336167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-09T20:09:43.952944+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4e7e57ca-2626-45fa-9a52-a35034d753ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6801","nonce":3125497649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6806","nonce":3125497649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6813","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6815","nonce":3125497649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6809","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6811","nonce":3125497649}]},"public_addr":"172.21.15.63:6801/3125497649","cluster_addr":"172.21.15.63:6806/3125497649","heartbeat_back_addr":"172.21.15.63:6815/3125497649","heartbeat_front_addr":"172.21.15.63:6811/3125497649","state":["exists","up"]},{"osd":1,"uuid":"442dbd50-2f65-4502-90fa-f126029d0c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6803","nonce":3768131182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6805","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6807","nonce":3768131182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6814","nonce":3768131182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6810","nonce":3768131182}]},"public_addr":"172.21.15.63:6803/3768131182","cluster_addr":"172.21.15.63:6807/3768131182","heartbeat_back_addr":"172.21.15.63:6814/3768131182","heartbeat_front_addr":"172.21.15.63:6810/3768131182","state":["exists","up"]},{"osd":2,"uuid":"5fed96b2-d11d-4f7b-9b07-9b43d41614d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6817","nonce":1087089498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6819","nonce":1087089498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6823","nonce":1087089498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6821","nonce":1087089498}]},"public_addr":"172.21.15.63:6817/1087089498","cluster_addr":"172.21.15.63:6819/1087089498","heartbeat_back_addr":"172.21.15.63:6823/1087089498","heartbeat_front_addr":"172.21.15.63:6821/1087089498","state":["exists","up"]},{"osd":3,"uuid":"a5c1635f-89ab-42aa-ad1e-f14da99b91b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6825","nonce":1568250839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6827","nonce":1568250839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6831","nonce":1568250839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6829","nonce":1568250839}]},"public_addr":"172.21.15.63:6825/1568250839","cluster_addr":"172.21.15.63:6827/1568250839","heartbeat_back_addr":"172.21.15.63:6831/1568250839","heartbeat_front_addr":"172.21.15.63:6829/1568250839","state":["exists","up"]},{"osd":4,"uuid":"62a4529d-709e-4eb6-951c-1cf3ab63d84a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6801","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6803","nonce":982132549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6810","nonce":982132549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6828","nonce":982132549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6818","nonce":982132549}]},"public_addr":"172.21.15.120:6803/982132549","cluster_addr":"172.21.15.120:6810/982132549","heartbeat_back_addr":"172.21.15.120:6828/982132549","heartbeat_front_addr":"172.21.15.120:6818/982132549","state":["exists","up"]},{"osd":5,"uuid":"cb37c5ed-ec5c-4ac2-af3f-9e16c200807e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6809","nonce":1971324730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6813","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6816","nonce":1971324730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6830","nonce":1971324730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6819","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6822","nonce":1971324730}]},"public_addr":"172.21.15.120:6809/1971324730","cluster_addr":"172.21.15.120:6816/1971324730","heartbeat_back_addr":"172.21.15.120:6830/1971324730","heartbeat_front_addr":"172.21.15.120:6822/1971324730","state":["exists","up"]},{"osd":6,"uuid":"d31c5357-5fcf-4da6-9e72-2f0f7f036acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6807","nonce":610448732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6817","nonce":610448732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6829","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6831","nonce":610448732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6821","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6825","nonce":610448732}]},"public_addr":"172.21.15.120:6807/610448732","cluster_addr":"172.21.15.120:6817/610448732","heartbeat_back_addr":"172.21.15.120:6831/610448732","heartbeat_front_addr":"172.21.15.120:6825/610448732","state":["exists","up"]},{"osd":7,"uuid":"5f957a83-94b2-4302-a0e3-f77757f341e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6805","nonce":3964176807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6811","nonce":3964176807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6827","nonce":3964176807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6820","nonce":3964176807}]},"public_addr":"172.21.15.120:6805/3964176807","cluster_addr":"172.21.15.120:6811/3964176807","heartbeat_back_addr":"172.21.15.120:6827/3964176807","heartbeat_front_addr":"172.21.15.120:6820/3964176807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.456405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.556324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.583822+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.539726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.353298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.493080+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.420249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.344365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T20:09:46.425 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T20:09:46.426 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T20:09:46.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:09:46.728 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":19,"fsid":"9ed7a651-94d4-45eb-abce-d57b88f09598","created":"2024-05-09T20:09:25.359123+0000","modified":"2024-05-09T20:09:46.699032+0000","last_up_change":"2024-05-09T20:09:37.441738+0000","last_in_change":"2024-05-09T20:09:27.861516+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T20:09:38.336167+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-09T20:09:43.952944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4e7e57ca-2626-45fa-9a52-a35034d753ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6801","nonce":3125497649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6806","nonce":3125497649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6813","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6815","nonce":3125497649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6809","nonce":3125497649},{"type":"v1","addr":"172.21.15.63:6811","nonce":3125497649}]},"public_addr":"172.21.15.63:6801/3125497649","cluster_addr":"172.21.15.63:6806/3125497649","heartbeat_back_addr":"172.21.15.63:6815/3125497649","heartbeat_front_addr":"172.21.15.63:6811/3125497649","state":["exists","up"]},{"osd":1,"uuid":"442dbd50-2f65-4502-90fa-f126029d0c95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6803","nonce":3768131182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6805","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6807","nonce":3768131182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6814","nonce":3768131182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3768131182},{"type":"v1","addr":"172.21.15.63:6810","nonce":3768131182}]},"public_addr":"172.21.15.63:6803/3768131182","cluster_addr":"172.21.15.63:6807/3768131182","heartbeat_back_addr":"172.21.15.63:6814/3768131182","heartbeat_front_addr":"172.21.15.63:6810/3768131182","state":["exists","up"]},{"osd":2,"uuid":"5fed96b2-d11d-4f7b-9b07-9b43d41614d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6817","nonce":1087089498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6819","nonce":1087089498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6823","nonce":1087089498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":1087089498},{"type":"v1","addr":"172.21.15.63:6821","nonce":1087089498}]},"public_addr":"172.21.15.63:6817/1087089498","cluster_addr":"172.21.15.63:6819/1087089498","heartbeat_back_addr":"172.21.15.63:6823/1087089498","heartbeat_front_addr":"172.21.15.63:6821/1087089498","state":["exists","up"]},{"osd":3,"uuid":"a5c1635f-89ab-42aa-ad1e-f14da99b91b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6825","nonce":1568250839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6827","nonce":1568250839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6830","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6831","nonce":1568250839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1568250839},{"type":"v1","addr":"172.21.15.63:6829","nonce":1568250839}]},"public_addr":"172.21.15.63:6825/1568250839","cluster_addr":"172.21.15.63:6827/1568250839","heartbeat_back_addr":"172.21.15.63:6831/1568250839","heartbeat_front_addr":"172.21.15.63:6829/1568250839","state":["exists","up"]},{"osd":4,"uuid":"62a4529d-709e-4eb6-951c-1cf3ab63d84a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6801","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6803","nonce":982132549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6810","nonce":982132549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6828","nonce":982132549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":982132549},{"type":"v1","addr":"172.21.15.120:6818","nonce":982132549}]},"public_addr":"172.21.15.120:6803/982132549","cluster_addr":"172.21.15.120:6810/982132549","heartbeat_back_addr":"172.21.15.120:6828/982132549","heartbeat_front_addr":"172.21.15.120:6818/982132549","state":["exists","up"]},{"osd":5,"uuid":"cb37c5ed-ec5c-4ac2-af3f-9e16c200807e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6809","nonce":1971324730}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6813","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6816","nonce":1971324730}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6830","nonce":1971324730}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6819","nonce":1971324730},{"type":"v1","addr":"172.21.15.120:6822","nonce":1971324730}]},"public_addr":"172.21.15.120:6809/1971324730","cluster_addr":"172.21.15.120:6816/1971324730","heartbeat_back_addr":"172.21.15.120:6830/1971324730","heartbeat_front_addr":"172.21.15.120:6822/1971324730","state":["exists","up"]},{"osd":6,"uuid":"d31c5357-5fcf-4da6-9e72-2f0f7f036acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6807","nonce":610448732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6817","nonce":610448732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6829","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6831","nonce":610448732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6821","nonce":610448732},{"type":"v1","addr":"172.21.15.120:6825","nonce":610448732}]},"public_addr":"172.21.15.120:6807/610448732","cluster_addr":"172.21.15.120:6817/610448732","heartbeat_back_addr":"172.21.15.120:6831/610448732","heartbeat_front_addr":"172.21.15.120:6825/610448732","state":["exists","up"]},{"osd":7,"uuid":"5f957a83-94b2-4302-a0e3-f77757f341e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6805","nonce":3964176807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6811","nonce":3964176807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6827","nonce":3964176807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":3964176807},{"type":"v1","addr":"172.21.15.120:6820","nonce":3964176807}]},"public_addr":"172.21.15.120:6805/3964176807","cluster_addr":"172.21.15.120:6811/3964176807","heartbeat_back_addr":"172.21.15.120:6827/3964176807","heartbeat_front_addr":"172.21.15.120:6820/3964176807","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.456405+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.556324+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.583822+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.539726+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.353298+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.493080+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.420249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T20:09:33.344365+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T20:09:46.748 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T20:09:46.748 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T20:09:46.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T20:09:46.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T20:09:46.749 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T20:09:46.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T20:09:46.750 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T20:09:46.751 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T20:09:47.009 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:47.009 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T20:09:47.018 INFO:teuthology.orchestra.run.smithi063.stdout:55834574854 2024-05-09T20:09:47.018 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T20:09:47.023 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:47.024 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T20:09:47.025 INFO:teuthology.orchestra.run.smithi063.stdout:55834574854 2024-05-09T20:09:47.025 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T20:09:47.026 INFO:teuthology.orchestra.run.smithi063.stdout:55834574852 2024-05-09T20:09:47.026 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T20:09:47.027 INFO:teuthology.orchestra.run.smithi063.stdout:55834574852 2024-05-09T20:09:47.028 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T20:09:47.029 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:47.030 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T20:09:47.031 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:47.031 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T20:09:47.499 INFO:teuthology.orchestra.run.smithi063.stdout:55834574852 2024-05-09T20:09:47.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-05-09T20:09:47.535 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:47.550 INFO:teuthology.orchestra.run.smithi063.stdout:55834574852 2024-05-09T20:09:47.553 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:47.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-05-09T20:09:47.572 INFO:teuthology.orchestra.run.smithi063.stdout:55834574851 2024-05-09T20:09:47.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2024-05-09T20:09:47.581 INFO:teuthology.orchestra.run.smithi063.stdout:55834574852 2024-05-09T20:09:47.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-05-09T20:09:47.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2024-05-09T20:09:47.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-05-09T20:09:47.611 INFO:teuthology.orchestra.run.smithi063.stdout:55834574851 2024-05-09T20:09:47.622 INFO:teuthology.orchestra.run.smithi063.stdout:55834574852 2024-05-09T20:09:47.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2024-05-09T20:09:47.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-05-09T20:09:48.523 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T20:09:48.563 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T20:09:48.576 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T20:09:48.584 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T20:09:48.592 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T20:09:48.601 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T20:09:48.630 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T20:09:48.640 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T20:09:49.024 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:49.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-05-09T20:09:49.047 DEBUG:teuthology.parallel:result is None 2024-05-09T20:09:49.101 INFO:teuthology.orchestra.run.smithi063.stdout:55834574854 2024-05-09T20:09:49.115 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:49.124 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:49.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-05-09T20:09:49.129 DEBUG:teuthology.parallel:result is None 2024-05-09T20:09:49.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-05-09T20:09:49.144 DEBUG:teuthology.parallel:result is None 2024-05-09T20:09:49.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-05-09T20:09:49.145 DEBUG:teuthology.parallel:result is None 2024-05-09T20:09:49.149 INFO:teuthology.orchestra.run.smithi063.stdout:55834574854 2024-05-09T20:09:49.151 INFO:teuthology.orchestra.run.smithi063.stdout:55834574853 2024-05-09T20:09:49.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-05-09T20:09:49.168 INFO:teuthology.orchestra.run.smithi063.stdout:55834574852 2024-05-09T20:09:49.168 DEBUG:teuthology.parallel:result is None 2024-05-09T20:09:49.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-05-09T20:09:49.180 DEBUG:teuthology.parallel:result is None 2024-05-09T20:09:49.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-05-09T20:09:49.185 DEBUG:teuthology.parallel:result is None 2024-05-09T20:09:49.187 INFO:teuthology.orchestra.run.smithi063.stdout:55834574852 2024-05-09T20:09:49.204 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-05-09T20:09:49.204 DEBUG:teuthology.parallel:result is None 2024-05-09T20:09:49.205 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T20:09:49.205 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T20:09:49.523 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:09:49.523 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-09T20:09:49.539 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-05-09T20:09:48.300133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218760,"kb_used_data":2568,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755960,"statfs":{"total":773094113280,"available":772870103040,"internally_reserved":0,"allocated":2629632,"data_stored":1639264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000908"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.718319+0000","last_change":"2024-05-09T20:09:45.716305+0000","last_active":"2024-05-09T20:09:45.718319+0000","last_peered":"2024-05-09T20:09:45.718319+0000","last_clean":"2024-05-09T20:09:45.718319+0000","last_became_active":"2024-05-09T20:09:45.715169+0000","last_became_peered":"2024-05-09T20:09:45.715169+0000","last_unstale":"2024-05-09T20:09:45.718319+0000","last_undegraded":"2024-05-09T20:09:45.718319+0000","last_fullsized":"2024-05-09T20:09:45.718319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:53:55.900014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T20:09:46.709914+0000","last_change":"2024-05-09T20:09:45.708863+0000","last_active":"2024-05-09T20:09:46.709914+0000","last_peered":"2024-05-09T20:09:46.709914+0000","last_clean":"2024-05-09T20:09:46.709914+0000","last_became_active":"2024-05-09T20:09:45.708060+0000","last_became_peered":"2024-05-09T20:09:45.708060+0000","last_unstale":"2024-05-09T20:09:46.709914+0000","last_undegraded":"2024-05-09T20:09:46.709914+0000","last_fullsized":"2024-05-09T20:09:46.709914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:52.760157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.883205+0000","last_change":"2024-05-09T20:09:45.882085+0000","last_active":"2024-05-09T20:09:45.883205+0000","last_peered":"2024-05-09T20:09:45.883205+0000","last_clean":"2024-05-09T20:09:45.883205+0000","last_became_active":"2024-05-09T20:09:45.881325+0000","last_became_peered":"2024-05-09T20:09:45.881325+0000","last_unstale":"2024-05-09T20:09:45.883205+0000","last_undegraded":"2024-05-09T20:09:45.883205+0000","last_fullsized":"2024-05-09T20:09:45.883205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T00:55:07.658774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T20:09:46.709640+0000","last_change":"2024-05-09T20:09:45.880171+0000","last_active":"2024-05-09T20:09:46.709640+0000","last_peered":"2024-05-09T20:09:46.709640+0000","last_clean":"2024-05-09T20:09:46.709640+0000","last_became_active":"2024-05-09T20:09:45.879805+0000","last_became_peered":"2024-05-09T20:09:45.879805+0000","last_unstale":"2024-05-09T20:09:46.709640+0000","last_undegraded":"2024-05-09T20:09:46.709640+0000","last_fullsized":"2024-05-09T20:09:46.709640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:17:48.033689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.718187+0000","last_change":"2024-05-09T20:09:45.715506+0000","last_active":"2024-05-09T20:09:45.718187+0000","last_peered":"2024-05-09T20:09:45.718187+0000","last_clean":"2024-05-09T20:09:45.718187+0000","last_became_active":"2024-05-09T20:09:45.714699+0000","last_became_peered":"2024-05-09T20:09:45.714699+0000","last_unstale":"2024-05-09T20:09:45.718187+0000","last_undegraded":"2024-05-09T20:09:45.718187+0000","last_fullsized":"2024-05-09T20:09:45.718187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T01:34:53.426422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.714331+0000","last_change":"2024-05-09T20:09:45.713230+0000","last_active":"2024-05-09T20:09:45.714331+0000","last_peered":"2024-05-09T20:09:45.714331+0000","last_clean":"2024-05-09T20:09:45.714331+0000","last_became_active":"2024-05-09T20:09:45.712768+0000","last_became_peered":"2024-05-09T20:09:45.712768+0000","last_unstale":"2024-05-09T20:09:45.714331+0000","last_undegraded":"2024-05-09T20:09:45.714331+0000","last_fullsized":"2024-05-09T20:09:45.714331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T01:02:28.562720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.719003+0000","last_change":"2024-05-09T20:09:45.717752+0000","last_active":"2024-05-09T20:09:45.719003+0000","last_peered":"2024-05-09T20:09:45.719003+0000","last_clean":"2024-05-09T20:09:45.719003+0000","last_became_active":"2024-05-09T20:09:45.716894+0000","last_became_peered":"2024-05-09T20:09:45.716894+0000","last_unstale":"2024-05-09T20:09:45.719003+0000","last_undegraded":"2024-05-09T20:09:45.719003+0000","last_fullsized":"2024-05-09T20:09:45.719003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:00.112854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.718203+0000","last_change":"2024-05-09T20:09:45.715991+0000","last_active":"2024-05-09T20:09:45.718203+0000","last_peered":"2024-05-09T20:09:45.718203+0000","last_clean":"2024-05-09T20:09:45.718203+0000","last_became_active":"2024-05-09T20:09:45.715222+0000","last_became_peered":"2024-05-09T20:09:45.715222+0000","last_unstale":"2024-05-09T20:09:45.718203+0000","last_undegraded":"2024-05-09T20:09:45.718203+0000","last_fullsized":"2024-05-09T20:09:45.718203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:55.641792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.712653+0000","last_change":"2024-05-09T20:09:39.559979+0000","last_active":"2024-05-09T20:09:45.712653+0000","last_peered":"2024-05-09T20:09:45.712653+0000","last_clean":"2024-05-09T20:09:45.712653+0000","last_became_active":"2024-05-09T20:09:39.557906+0000","last_became_peered":"2024-05-09T20:09:39.557906+0000","last_unstale":"2024-05-09T20:09:45.712653+0000","last_undegraded":"2024-05-09T20:09:45.712653+0000","last_fullsized":"2024-05-09T20:09:45.712653+0000","mapping_epoch":14,"log_start":"15'100","ondisk_log_start":"15'100","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:38.469635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:38.469635+0000","last_clean_scrub_stamp":"2024-05-09T20:09:38.469635+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:46.838604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T20:09:49.540 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T20:09:49.819 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:09:49.820 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-05-09T20:09:49.836 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-05-09T20:09:48.300133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218760,"kb_used_data":2568,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755960,"statfs":{"total":773094113280,"available":772870103040,"internally_reserved":0,"allocated":2629632,"data_stored":1639264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000908"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.718319+0000","last_change":"2024-05-09T20:09:45.716305+0000","last_active":"2024-05-09T20:09:45.718319+0000","last_peered":"2024-05-09T20:09:45.718319+0000","last_clean":"2024-05-09T20:09:45.718319+0000","last_became_active":"2024-05-09T20:09:45.715169+0000","last_became_peered":"2024-05-09T20:09:45.715169+0000","last_unstale":"2024-05-09T20:09:45.718319+0000","last_undegraded":"2024-05-09T20:09:45.718319+0000","last_fullsized":"2024-05-09T20:09:45.718319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T04:53:55.900014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T20:09:46.709914+0000","last_change":"2024-05-09T20:09:45.708863+0000","last_active":"2024-05-09T20:09:46.709914+0000","last_peered":"2024-05-09T20:09:46.709914+0000","last_clean":"2024-05-09T20:09:46.709914+0000","last_became_active":"2024-05-09T20:09:45.708060+0000","last_became_peered":"2024-05-09T20:09:45.708060+0000","last_unstale":"2024-05-09T20:09:46.709914+0000","last_undegraded":"2024-05-09T20:09:46.709914+0000","last_fullsized":"2024-05-09T20:09:46.709914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T20:52:52.760157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.883205+0000","last_change":"2024-05-09T20:09:45.882085+0000","last_active":"2024-05-09T20:09:45.883205+0000","last_peered":"2024-05-09T20:09:45.883205+0000","last_clean":"2024-05-09T20:09:45.883205+0000","last_became_active":"2024-05-09T20:09:45.881325+0000","last_became_peered":"2024-05-09T20:09:45.881325+0000","last_unstale":"2024-05-09T20:09:45.883205+0000","last_undegraded":"2024-05-09T20:09:45.883205+0000","last_fullsized":"2024-05-09T20:09:45.883205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T00:55:07.658774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T20:09:46.709640+0000","last_change":"2024-05-09T20:09:45.880171+0000","last_active":"2024-05-09T20:09:46.709640+0000","last_peered":"2024-05-09T20:09:46.709640+0000","last_clean":"2024-05-09T20:09:46.709640+0000","last_became_active":"2024-05-09T20:09:45.879805+0000","last_became_peered":"2024-05-09T20:09:45.879805+0000","last_unstale":"2024-05-09T20:09:46.709640+0000","last_undegraded":"2024-05-09T20:09:46.709640+0000","last_fullsized":"2024-05-09T20:09:46.709640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T22:17:48.033689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.718187+0000","last_change":"2024-05-09T20:09:45.715506+0000","last_active":"2024-05-09T20:09:45.718187+0000","last_peered":"2024-05-09T20:09:45.718187+0000","last_clean":"2024-05-09T20:09:45.718187+0000","last_became_active":"2024-05-09T20:09:45.714699+0000","last_became_peered":"2024-05-09T20:09:45.714699+0000","last_unstale":"2024-05-09T20:09:45.718187+0000","last_undegraded":"2024-05-09T20:09:45.718187+0000","last_fullsized":"2024-05-09T20:09:45.718187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T01:34:53.426422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.714331+0000","last_change":"2024-05-09T20:09:45.713230+0000","last_active":"2024-05-09T20:09:45.714331+0000","last_peered":"2024-05-09T20:09:45.714331+0000","last_clean":"2024-05-09T20:09:45.714331+0000","last_became_active":"2024-05-09T20:09:45.712768+0000","last_became_peered":"2024-05-09T20:09:45.712768+0000","last_unstale":"2024-05-09T20:09:45.714331+0000","last_undegraded":"2024-05-09T20:09:45.714331+0000","last_fullsized":"2024-05-09T20:09:45.714331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T01:02:28.562720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.719003+0000","last_change":"2024-05-09T20:09:45.717752+0000","last_active":"2024-05-09T20:09:45.719003+0000","last_peered":"2024-05-09T20:09:45.719003+0000","last_clean":"2024-05-09T20:09:45.719003+0000","last_became_active":"2024-05-09T20:09:45.716894+0000","last_became_peered":"2024-05-09T20:09:45.716894+0000","last_unstale":"2024-05-09T20:09:45.719003+0000","last_undegraded":"2024-05-09T20:09:45.719003+0000","last_fullsized":"2024-05-09T20:09:45.719003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T06:45:00.112854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.718203+0000","last_change":"2024-05-09T20:09:45.715991+0000","last_active":"2024-05-09T20:09:45.718203+0000","last_peered":"2024-05-09T20:09:45.718203+0000","last_clean":"2024-05-09T20:09:45.718203+0000","last_became_active":"2024-05-09T20:09:45.715222+0000","last_became_peered":"2024-05-09T20:09:45.715222+0000","last_unstale":"2024-05-09T20:09:45.718203+0000","last_undegraded":"2024-05-09T20:09:45.718203+0000","last_fullsized":"2024-05-09T20:09:45.718203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:44.679886+0000","last_clean_scrub_stamp":"2024-05-09T20:09:44.679886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:06:55.641792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-09T20:09:45.712653+0000","last_change":"2024-05-09T20:09:39.559979+0000","last_active":"2024-05-09T20:09:45.712653+0000","last_peered":"2024-05-09T20:09:45.712653+0000","last_clean":"2024-05-09T20:09:45.712653+0000","last_became_active":"2024-05-09T20:09:39.557906+0000","last_became_peered":"2024-05-09T20:09:39.557906+0000","last_unstale":"2024-05-09T20:09:45.712653+0000","last_undegraded":"2024-05-09T20:09:45.712653+0000","last_fullsized":"2024-05-09T20:09:45.712653+0000","mapping_epoch":14,"log_start":"15'100","ondisk_log_start":"15'100","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T20:09:38.469635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T20:09:38.469635+0000","last_clean_scrub_stamp":"2024-05-09T20:09:38.469635+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":100,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-11T05:32:46.838604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":180224,"data_stored":57316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27780,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344060,"statfs":{"total":96636764160,"available":96608317440,"internally_reserved":0,"allocated":774144,"data_stored":647684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T20:09:49.836 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T20:09:49.837 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T20:09:49.837 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-09T20:09:49.837 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T20:09:50.150 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:09:50.150 INFO:teuthology.orchestra.run.smithi063.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T20:09:50.168 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-09T20:09:50.168 INFO:teuthology.run_tasks:Running task openssl_keys... 2024-05-09T20:09:50.178 INFO:teuthology.run_tasks:Running task rgw... 2024-05-09T20:09:50.232 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2024-05-09T20:09:50.232 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-05-09T20:09:50.232 INFO:tasks.rgw:Creating data pools 2024-05-09T20:09:50.232 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-05-09T20:09:50.232 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd erasure-code-profile set client.0 k=2 m=1 crush-failure-domain=osd --cluster ceph 2024-05-09T20:09:50.743 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-05-09T20:09:52.736 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'default.rgw.buckets.data' created 2024-05-09T20:09:52.774 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-05-09T20:09:53.750 INFO:teuthology.orchestra.run.smithi063.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-05-09T20:09:53.800 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-05-09T20:09:54.743 INFO:teuthology.orchestra.run.smithi063.stderr:pool 'default.rgw.buckets.index' created 2024-05-09T20:09:54.780 DEBUG:teuthology.orchestra.run.smithi063:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-05-09T20:09:55.779 INFO:teuthology.orchestra.run.smithi063.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-05-09T20:09:55.814 DEBUG:tasks.rgw:Pools created 2024-05-09T20:09:55.814 INFO:tasks.rgw:Configuring compression type = random 2024-05-09T20:09:55.814 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-05-09T20:09:55.814 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-05-09T20:09:55.815 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-05-09T20:09:55.871 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:09:55.872 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:09:58.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.055+0000 7f973d6b3640 0 -- 172.21.15.63:0/2592263615 >> [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] conn(0x7f9618066c70 msgr2=0x7f9618069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:09:58.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.063+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973c6b1640 0 -- 172.21.15.63:0/967217389 >> [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] conn(0x7f9604066d60 msgr2=0x7f9604069220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:09:58.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973f39e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:09:58.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973f39e980 20 realm 2024-05-09T20:09:58.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973f39e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:09:58.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973f39e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:09:58.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973f39e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:09:58.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.067+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.071+0000 7f973f39e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:09:58.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.071+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.071+0000 7f973f39e980 20 rados_obj.operate() r=0 bl.length=993 2024-05-09T20:09:58.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.071+0000 7f973f39e980 20 searching for the correct realm 2024-05-09T20:09:58.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:09:58.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:09:58.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:09:58.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:09:58.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:09:58.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:09:58.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:09:58.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.095+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:09:58.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 20 rados_obj.operate() r=0 bl.length=436 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 20 zone default found 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 4 Realm: () 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 20 zonegroup default 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:09:58.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:58.099+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:59.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:59.795+0000 7f973f39e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:09:59.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:59.795+0000 7f973f39e980 20 rados->read ofs=0 len=0 2024-05-09T20:09:59.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:59.799+0000 7f973f39e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:09:59.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:59.799+0000 7f973f39e980 20 started sync module instance, tier type = 2024-05-09T20:09:59.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:09:59.799+0000 7f973f39e980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:10:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 20 add_watcher() i=0 2024-05-09T20:10:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 20 add_watcher() i=1 2024-05-09T20:10:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 20 add_watcher() i=2 2024-05-09T20:10:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 20 add_watcher() i=3 2024-05-09T20:10:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 20 add_watcher() i=4 2024-05-09T20:10:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 20 add_watcher() i=5 2024-05-09T20:10:01.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 20 add_watcher() i=6 2024-05-09T20:10:01.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 20 add_watcher() i=7 2024-05-09T20:10:01.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.839+0000 7f973f39e980 2 all 8 watchers are set, enabling cache 2024-05-09T20:10:01.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 5 tl::expected, obj_version>, boost::system::error_code> logback_generations::read(const DoutPrefixProvider*, optional_yield):410: oid=data_loggenerations_metadata not found 2024-05-09T20:10:01.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.0 2024-05-09T20:10:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.0 does not exist 2024-05-09T20:10:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.1 2024-05-09T20:10:01.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.1 does not exist 2024-05-09T20:10:01.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.2 2024-05-09T20:10:01.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.2 does not exist 2024-05-09T20:10:01.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.3 2024-05-09T20:10:01.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.3 does not exist 2024-05-09T20:10:01.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.4 2024-05-09T20:10:01.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.4 does not exist 2024-05-09T20:10:01.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.5 2024-05-09T20:10:01.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.843+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.5 does not exist 2024-05-09T20:10:01.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.6 2024-05-09T20:10:01.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.6 does not exist 2024-05-09T20:10:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.7 2024-05-09T20:10:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.7 does not exist 2024-05-09T20:10:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.8 2024-05-09T20:10:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.8 does not exist 2024-05-09T20:10:01.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.9 2024-05-09T20:10:01.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.9 does not exist 2024-05-09T20:10:01.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.10 2024-05-09T20:10:01.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.10 does not exist 2024-05-09T20:10:01.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.11 2024-05-09T20:10:01.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.11 does not exist 2024-05-09T20:10:01.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.12 2024-05-09T20:10:01.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.12 does not exist 2024-05-09T20:10:01.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.13 2024-05-09T20:10:01.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.847+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.13 does not exist 2024-05-09T20:10:01.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.14 2024-05-09T20:10:01.845 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.14 does not exist 2024-05-09T20:10:01.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.15 2024-05-09T20:10:01.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.15 does not exist 2024-05-09T20:10:01.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.16 2024-05-09T20:10:01.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.16 does not exist 2024-05-09T20:10:01.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.17 2024-05-09T20:10:01.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.17 does not exist 2024-05-09T20:10:01.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.18 2024-05-09T20:10:01.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.18 does not exist 2024-05-09T20:10:01.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.19 2024-05-09T20:10:01.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.851+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.19 does not exist 2024-05-09T20:10:01.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.20 2024-05-09T20:10:01.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.20 does not exist 2024-05-09T20:10:01.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.21 2024-05-09T20:10:01.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.21 does not exist 2024-05-09T20:10:01.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.22 2024-05-09T20:10:01.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.22 does not exist 2024-05-09T20:10:01.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.23 2024-05-09T20:10:01.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.23 does not exist 2024-05-09T20:10:01.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.24 2024-05-09T20:10:01.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.24 does not exist 2024-05-09T20:10:01.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.25 2024-05-09T20:10:01.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.855+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.25 does not exist 2024-05-09T20:10:01.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.26 2024-05-09T20:10:01.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.26 does not exist 2024-05-09T20:10:01.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.27 2024-05-09T20:10:01.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.27 does not exist 2024-05-09T20:10:01.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.28 2024-05-09T20:10:01.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.28 does not exist 2024-05-09T20:10:01.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.29 2024-05-09T20:10:01.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.29 does not exist 2024-05-09T20:10:01.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.30 2024-05-09T20:10:01.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.30 does not exist 2024-05-09T20:10:01.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.31 2024-05-09T20:10:01.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.859+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.31 does not exist 2024-05-09T20:10:01.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.32 2024-05-09T20:10:01.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.32 does not exist 2024-05-09T20:10:01.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.33 2024-05-09T20:10:01.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.33 does not exist 2024-05-09T20:10:01.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.34 2024-05-09T20:10:01.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.34 does not exist 2024-05-09T20:10:01.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.35 2024-05-09T20:10:01.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.35 does not exist 2024-05-09T20:10:01.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.36 2024-05-09T20:10:01.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.36 does not exist 2024-05-09T20:10:01.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.37 2024-05-09T20:10:01.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.863+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.37 does not exist 2024-05-09T20:10:01.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.38 2024-05-09T20:10:01.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.38 does not exist 2024-05-09T20:10:01.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.39 2024-05-09T20:10:01.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.39 does not exist 2024-05-09T20:10:01.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.40 2024-05-09T20:10:01.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.40 does not exist 2024-05-09T20:10:01.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.41 2024-05-09T20:10:01.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.41 does not exist 2024-05-09T20:10:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.42 2024-05-09T20:10:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.42 does not exist 2024-05-09T20:10:01.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.43 2024-05-09T20:10:01.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.867+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.43 does not exist 2024-05-09T20:10:01.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.44 2024-05-09T20:10:01.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.44 does not exist 2024-05-09T20:10:01.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.45 2024-05-09T20:10:01.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.45 does not exist 2024-05-09T20:10:01.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.46 2024-05-09T20:10:01.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.46 does not exist 2024-05-09T20:10:01.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.47 2024-05-09T20:10:01.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.47 does not exist 2024-05-09T20:10:01.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.48 2024-05-09T20:10:01.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.48 does not exist 2024-05-09T20:10:01.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.49 2024-05-09T20:10:01.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.871+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.49 does not exist 2024-05-09T20:10:01.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.50 2024-05-09T20:10:01.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.50 does not exist 2024-05-09T20:10:01.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.51 2024-05-09T20:10:01.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.51 does not exist 2024-05-09T20:10:01.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.52 2024-05-09T20:10:01.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.52 does not exist 2024-05-09T20:10:01.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.53 2024-05-09T20:10:01.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.53 does not exist 2024-05-09T20:10:01.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.54 2024-05-09T20:10:01.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.54 does not exist 2024-05-09T20:10:01.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.55 2024-05-09T20:10:01.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.875+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.55 does not exist 2024-05-09T20:10:01.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.56 2024-05-09T20:10:01.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.56 does not exist 2024-05-09T20:10:01.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.57 2024-05-09T20:10:01.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.57 does not exist 2024-05-09T20:10:01.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.58 2024-05-09T20:10:01.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.58 does not exist 2024-05-09T20:10:01.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.59 2024-05-09T20:10:01.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.59 does not exist 2024-05-09T20:10:01.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.60 2024-05-09T20:10:01.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.60 does not exist 2024-05-09T20:10:01.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.61 2024-05-09T20:10:01.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.61 does not exist 2024-05-09T20:10:01.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.62 2024-05-09T20:10:01.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.879+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.62 does not exist 2024-05-09T20:10:01.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.63 2024-05-09T20:10:01.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.63 does not exist 2024-05-09T20:10:01.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.64 2024-05-09T20:10:01.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.64 does not exist 2024-05-09T20:10:01.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.65 2024-05-09T20:10:01.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.65 does not exist 2024-05-09T20:10:01.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.66 2024-05-09T20:10:01.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.66 does not exist 2024-05-09T20:10:01.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.67 2024-05-09T20:10:01.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.67 does not exist 2024-05-09T20:10:01.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.68 2024-05-09T20:10:01.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.68 does not exist 2024-05-09T20:10:01.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.69 2024-05-09T20:10:01.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.69 does not exist 2024-05-09T20:10:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.70 2024-05-09T20:10:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.883+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.70 does not exist 2024-05-09T20:10:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.71 2024-05-09T20:10:01.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.71 does not exist 2024-05-09T20:10:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.72 2024-05-09T20:10:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.72 does not exist 2024-05-09T20:10:01.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.73 2024-05-09T20:10:01.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.73 does not exist 2024-05-09T20:10:01.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.74 2024-05-09T20:10:01.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.74 does not exist 2024-05-09T20:10:01.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.75 2024-05-09T20:10:01.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.75 does not exist 2024-05-09T20:10:01.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.76 2024-05-09T20:10:01.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.887+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.76 does not exist 2024-05-09T20:10:01.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.77 2024-05-09T20:10:01.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.77 does not exist 2024-05-09T20:10:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.78 2024-05-09T20:10:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.78 does not exist 2024-05-09T20:10:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.79 2024-05-09T20:10:01.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.79 does not exist 2024-05-09T20:10:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.80 2024-05-09T20:10:01.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.80 does not exist 2024-05-09T20:10:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.81 2024-05-09T20:10:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.81 does not exist 2024-05-09T20:10:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.82 2024-05-09T20:10:01.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.891+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.82 does not exist 2024-05-09T20:10:01.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.83 2024-05-09T20:10:01.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.83 does not exist 2024-05-09T20:10:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.84 2024-05-09T20:10:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.84 does not exist 2024-05-09T20:10:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.85 2024-05-09T20:10:01.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.85 does not exist 2024-05-09T20:10:01.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.86 2024-05-09T20:10:01.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.86 does not exist 2024-05-09T20:10:01.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.87 2024-05-09T20:10:01.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.87 does not exist 2024-05-09T20:10:01.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.88 2024-05-09T20:10:01.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.895+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.88 does not exist 2024-05-09T20:10:01.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.89 2024-05-09T20:10:01.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.89 does not exist 2024-05-09T20:10:01.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.90 2024-05-09T20:10:01.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.90 does not exist 2024-05-09T20:10:01.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.91 2024-05-09T20:10:01.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.91 does not exist 2024-05-09T20:10:01.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.92 2024-05-09T20:10:01.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.92 does not exist 2024-05-09T20:10:01.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.93 2024-05-09T20:10:01.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.93 does not exist 2024-05-09T20:10:01.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.94 2024-05-09T20:10:01.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.899+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.94 does not exist 2024-05-09T20:10:01.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.95 2024-05-09T20:10:01.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.95 does not exist 2024-05-09T20:10:01.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.96 2024-05-09T20:10:01.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.96 does not exist 2024-05-09T20:10:01.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.97 2024-05-09T20:10:01.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.97 does not exist 2024-05-09T20:10:01.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.98 2024-05-09T20:10:01.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.98 does not exist 2024-05-09T20:10:01.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.99 2024-05-09T20:10:01.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.99 does not exist 2024-05-09T20:10:01.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.100 2024-05-09T20:10:01.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.903+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.100 does not exist 2024-05-09T20:10:01.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.101 2024-05-09T20:10:01.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.101 does not exist 2024-05-09T20:10:01.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.102 2024-05-09T20:10:01.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.102 does not exist 2024-05-09T20:10:01.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.103 2024-05-09T20:10:01.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.103 does not exist 2024-05-09T20:10:01.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.104 2024-05-09T20:10:01.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.104 does not exist 2024-05-09T20:10:01.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.105 2024-05-09T20:10:01.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.105 does not exist 2024-05-09T20:10:01.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.106 2024-05-09T20:10:01.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.907+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.106 does not exist 2024-05-09T20:10:01.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.107 2024-05-09T20:10:01.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.107 does not exist 2024-05-09T20:10:01.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.108 2024-05-09T20:10:01.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.108 does not exist 2024-05-09T20:10:01.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.109 2024-05-09T20:10:01.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.109 does not exist 2024-05-09T20:10:01.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.110 2024-05-09T20:10:01.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.110 does not exist 2024-05-09T20:10:01.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.111 2024-05-09T20:10:01.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.111 does not exist 2024-05-09T20:10:01.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.112 2024-05-09T20:10:01.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.911+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.112 does not exist 2024-05-09T20:10:01.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.113 2024-05-09T20:10:01.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.113 does not exist 2024-05-09T20:10:01.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.114 2024-05-09T20:10:01.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.114 does not exist 2024-05-09T20:10:01.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.115 2024-05-09T20:10:01.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.115 does not exist 2024-05-09T20:10:01.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.116 2024-05-09T20:10:01.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.116 does not exist 2024-05-09T20:10:01.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.117 2024-05-09T20:10:01.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.117 does not exist 2024-05-09T20:10:01.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.118 2024-05-09T20:10:01.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.915+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.118 does not exist 2024-05-09T20:10:01.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.119 2024-05-09T20:10:01.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.119 does not exist 2024-05-09T20:10:01.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.120 2024-05-09T20:10:01.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.120 does not exist 2024-05-09T20:10:01.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.121 2024-05-09T20:10:01.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.121 does not exist 2024-05-09T20:10:01.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.122 2024-05-09T20:10:01.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.122 does not exist 2024-05-09T20:10:01.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.123 2024-05-09T20:10:01.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.123 does not exist 2024-05-09T20:10:01.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.124 2024-05-09T20:10:01.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.124 does not exist 2024-05-09T20:10:01.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.125 2024-05-09T20:10:01.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.919+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.923+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.125 does not exist 2024-05-09T20:10:01.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.923+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.126 2024-05-09T20:10:01.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.923+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.923+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.126 does not exist 2024-05-09T20:10:01.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.923+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):38 probing oid=data_log.127 2024-05-09T20:10:01.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.923+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.923+0000 7f973f39e980 20 shard_check {anonymous}::probe_shard(const DoutPrefixProvider*, librados::v14_2_0::IoCtx&, const string&, bool&, optional_yield):60: oid=data_log.127 does not exist 2024-05-09T20:10:01.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.923+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::create(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, std::optional >, bool, uint64_t, uint64_t):1211 entering 2024-05-09T20:10:01.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.927+0000 7f973f39e980 20 static int rgw::cls::fifo::FIFO::open(const DoutPrefixProvider*, librados::v14_2_0::IoCtx, std::string, std::unique_ptr*, optional_yield, std::optional, bool):1169 entering 2024-05-09T20:10:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.935+0000 7f95eefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:10:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.935+0000 7f973f39e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:10:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:01.935+0000 7f973f39e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:10:04.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.803+0000 7f973f39e980 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2024-05-09T20:10:04.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.803+0000 7f973f39e980 5 note: GC not initialized 2024-05-09T20:10:04.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.819+0000 7f95edfe3640 20 reqs_thread_entry: start 2024-05-09T20:10:04.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.819+0000 7f973f39e980 20 init_complete bucket index max shards: 11 2024-05-09T20:10:04.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.819+0000 7f973f39e980 20 Filter name: none 2024-05-09T20:10:04.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.819+0000 7f95477fe640 20 reqs_thread_entry: start 2024-05-09T20:10:04.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.863+0000 7f95eefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:10:04.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.867+0000 7f973f39e980 20 remove_watcher() i=0 2024-05-09T20:10:04.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.867+0000 7f973f39e980 2 removed watcher, disabling cache 2024-05-09T20:10:04.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.871+0000 7f973f39e980 20 remove_watcher() i=1 2024-05-09T20:10:04.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.871+0000 7f973f39e980 20 remove_watcher() i=2 2024-05-09T20:10:04.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.879+0000 7f973f39e980 20 remove_watcher() i=3 2024-05-09T20:10:04.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.883+0000 7f973f39e980 20 remove_watcher() i=4 2024-05-09T20:10:04.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.883+0000 7f973f39e980 20 remove_watcher() i=5 2024-05-09T20:10:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.887+0000 7f973f39e980 20 remove_watcher() i=6 2024-05-09T20:10:04.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.887+0000 7f973f39e980 20 remove_watcher() i=7 2024-05-09T20:10:04.900 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-09T20:10:04.900 DEBUG:tasks.util.rgw: json result: [] 2024-05-09T20:10:04.900 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-05-09T20:10:04.901 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2024-05-09T20:10:04.901 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2024-05-09T20:10:04.943 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:10:04.943 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:10:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.979+0000 7f27f318b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:10:04.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.979+0000 7f27f318b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:10:04.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:04.983+0000 7f2636ffd640 20 reqs_thread_entry: start 2024-05-09T20:10:05.001 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-09T20:10:05.001 DEBUG:tasks.util.rgw: json result: {'id': '6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-09T20:10:05.002 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2024-05-09T20:10:05.002 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2024-05-09T20:10:05.002 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2024-05-09T20:10:05.073 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:10:05.073 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:10:05.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:10:05.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 realm 2024-05-09T20:10:05.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:10:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:10:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:10:05.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:10:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.119+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.123+0000 7f162a379980 20 rados_obj.operate() r=0 bl.length=1009 2024-05-09T20:10:05.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.123+0000 7f162a379980 20 searching for the correct realm 2024-05-09T20:10:05.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.123+0000 7f162768c640 0 -- 172.21.15.63:0/682914591 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f16180b9be0 msgr2=0x7f16180c5960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:10:05.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.131+0000 7f162768c640 0 -- 172.21.15.63:0/682914591 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f16180e3040 msgr2=0x7f16180e3930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:10:05.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:10:05.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:10:05.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:10:05.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:10:05.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:10:05.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:10:05.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:10:05.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:10:05.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 rados_obj.operate() r=0 bl.length=436 2024-05-09T20:10:05.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 zone default found 2024-05-09T20:10:05.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 4 Realm: () 2024-05-09T20:10:05.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:10:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:10:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:10:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 zonegroup default 2024-05-09T20:10:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.355+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.359+0000 7f162a379980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:10:05.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.359+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.359+0000 7f162a379980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:10:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.359+0000 7f162a379980 20 rados->read ofs=0 len=0 2024-05-09T20:10:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.359+0000 7f162a379980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:10:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.359+0000 7f162a379980 20 started sync module instance, tier type = 2024-05-09T20:10:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.359+0000 7f162a379980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:10:05.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.383+0000 7f162a379980 20 add_watcher() i=0 2024-05-09T20:10:05.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.383+0000 7f162a379980 20 add_watcher() i=1 2024-05-09T20:10:05.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.383+0000 7f162a379980 20 add_watcher() i=2 2024-05-09T20:10:05.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.383+0000 7f162a379980 20 add_watcher() i=3 2024-05-09T20:10:05.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.383+0000 7f162a379980 20 add_watcher() i=4 2024-05-09T20:10:05.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.383+0000 7f162a379980 20 add_watcher() i=5 2024-05-09T20:10:05.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.383+0000 7f162a379980 20 add_watcher() i=6 2024-05-09T20:10:05.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.387+0000 7f162a379980 20 add_watcher() i=7 2024-05-09T20:10:05.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.387+0000 7f162a379980 2 all 8 watchers are set, enabling cache 2024-05-09T20:10:05.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.391+0000 7f14ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:10:05.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.391+0000 7f162a379980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:10:05.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.391+0000 7f162a379980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:10:05.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.391+0000 7f162a379980 5 note: GC not initialized 2024-05-09T20:10:05.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.391+0000 7f14dcfe1640 20 reqs_thread_entry: start 2024-05-09T20:10:05.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.391+0000 7f162a379980 20 init_complete bucket index max shards: 11 2024-05-09T20:10:05.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.391+0000 7f162a379980 20 Filter name: none 2024-05-09T20:10:05.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.391+0000 7f14567fc640 20 reqs_thread_entry: start 2024-05-09T20:10:05.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.415+0000 7f14ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:10:05.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.419+0000 7f162a379980 20 remove_watcher() i=0 2024-05-09T20:10:05.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.419+0000 7f162a379980 2 removed watcher, disabling cache 2024-05-09T20:10:05.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.419+0000 7f162a379980 20 remove_watcher() i=1 2024-05-09T20:10:05.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.423+0000 7f162a379980 20 remove_watcher() i=2 2024-05-09T20:10:05.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.423+0000 7f162a379980 20 remove_watcher() i=3 2024-05-09T20:10:05.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.427+0000 7f162a379980 20 remove_watcher() i=4 2024-05-09T20:10:05.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.427+0000 7f162a379980 20 remove_watcher() i=5 2024-05-09T20:10:05.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.427+0000 7f162a379980 20 remove_watcher() i=6 2024-05-09T20:10:05.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.431+0000 7f162a379980 20 remove_watcher() i=7 2024-05-09T20:10:05.442 INFO:teuthology.orchestra.run.smithi063.stdout:[] 2024-05-09T20:10:05.442 DEBUG:tasks.util.rgw: json result: [] 2024-05-09T20:10:05.442 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2024-05-09T20:10:05.442 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-05-09T20:10:05.442 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2024-05-09T20:10:05.442 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2024-05-09T20:10:05.529 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:10:05.529 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:10:05.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.563+0000 7f3441b38980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:10:05.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.563+0000 7f3441b38980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:10:05.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.567+0000 7f32a5ffb640 20 reqs_thread_entry: start 2024-05-09T20:10:05.582 INFO:teuthology.orchestra.run.smithi063.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2024-05-09T20:10:05.583 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['LUKEWARM', 'STANDARD']}}] 2024-05-09T20:10:05.583 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-05-09T20:10:05.583 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2024-05-09T20:10:05.583 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2024-05-09T20:10:05.663 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:10:05.663 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:10:05.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.695+0000 7fd666a7b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:10:05.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.695+0000 7fd666a7b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:10:05.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.699+0000 7fd4ba7fc640 20 reqs_thread_entry: start 2024-05-09T20:10:05.717 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-09T20:10:05.717 DEBUG:tasks.util.rgw: json result: {'id': '6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-09T20:10:05.717 INFO:tasks.rgw:Configuring storage class type = FROZEN 2024-05-09T20:10:05.718 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-05-09T20:10:05.718 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2024-05-09T20:10:05.718 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2024-05-09T20:10:05.797 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:10:05.797 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:10:05.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.827+0000 7fba87201980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:10:05.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.827+0000 7fba87201980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:10:05.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.831+0000 7fb8caffd640 20 reqs_thread_entry: start 2024-05-09T20:10:05.847 INFO:teuthology.orchestra.run.smithi063.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2024-05-09T20:10:05.847 DEBUG:tasks.util.rgw: json result: [{'key': 'default-placement', 'val': {'name': 'default-placement', 'tags': [], 'storage_classes': ['FROZEN', 'LUKEWARM', 'STANDARD']}}] 2024-05-09T20:10:05.847 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-05-09T20:10:05.847 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2024-05-09T20:10:05.848 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2024-05-09T20:10:05.933 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:10:05.934 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:10:05.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.967+0000 7f98ecd34980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:10:05.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.967+0000 7f98ecd34980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:10:05.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:10:05.971+0000 7f97a8ff9640 20 reqs_thread_entry: start 2024-05-09T20:10:05.987 INFO:teuthology.orchestra.run.smithi063.stdout:{"id":"6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-09T20:10:05.987 DEBUG:tasks.util.rgw: json result: {'id': '6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'FROZEN': {'data_pool': 'default.rgw.buckets.data.frozen'}, 'LUKEWARM': {'data_pool': 'default.rgw.buckets.data.lukewarm'}, 'STANDARD': {'data_pool': 'default.rgw.buckets.data', 'compression_type': 'random'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-09T20:10:05.987 INFO:tasks.rgw:Starting rgw... 2024-05-09T20:10:05.987 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2024-05-09T20:10:05.987 INFO:tasks.rgw:Using beast as radosgw frontend 2024-05-09T20:10:05.988 DEBUG:teuthology.orchestra.run.smithi063:> sudo echo -n http://smithi063.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-05-09T20:10:05.998 INFO:teuthology.orchestra.run.smithi063.stdout:http://smithi063.front.sepia.ceph.com:80 2024-05-09T20:10:05.999 DEBUG:teuthology.orchestra.run.smithi063:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-05-09T20:10:06.051 DEBUG:tasks.ceph_manager:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--max-threads=1024'] 2024-05-09T20:10:06.051 INFO:tasks.rgw.client.0:Restarting daemon 2024-05-09T20:10:06.051 DEBUG:teuthology.orchestra.run.smithi063:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --vgdb=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-05-09T20:10:06.095 INFO:tasks.rgw.client.0:Started 2024-05-09T20:10:06.095 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi063.front.sepia.ceph.com:80/ 2024-05-09T20:10:06.095 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-05-09T20:10:06.111 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-09T20:10:06.112 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-09T20:10:06.112 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-05-09T20:10:06.112 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-09T20:10:06.112 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-09T20:10:07.113 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-05-09T20:10:07.125 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-09T20:10:07.125 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-05-09T20:10:07.125 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-09T20:10:07.126 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-09T20:10:07.126 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-09T20:10:09.127 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-05-09T20:10:09.138 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-09T20:10:09.139 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-09T20:10:09.139 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-05-09T20:10:09.139 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-09T20:10:09.139 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-09T20:10:13.140 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-05-09T20:10:13.151 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-09T20:10:13.151 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-05-09T20:10:13.151 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-09T20:10:13.151 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-09T20:10:13.152 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-09T20:10:21.153 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-05-09T20:10:21.163 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-09T20:10:21.163 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-05-09T20:10:21.163 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-09T20:10:21.163 INFO:teuthology.orchestra.run.smithi063.stderr:curl: (7) Failed to connect to smithi063.front.sepia.ceph.com port 80 after 0 ms: Connection refused 2024-05-09T20:10:21.164 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-09T20:10:31.098 INFO:tasks.rgw.client.0.smithi063.stdout:2024-05-09T20:10:30.791+0000 8e9ff80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:10:37.165 DEBUG:teuthology.orchestra.run.smithi063:> curl http://smithi063.front.sepia.ceph.com:80/ 2024-05-09T20:10:37.177 INFO:teuthology.orchestra.run.smithi063.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-09T20:10:37.177 INFO:teuthology.orchestra.run.smithi063.stderr: Dload Upload Total Spent Left Speed 2024-05-09T20:10:37.715 INFO:teuthology.orchestra.run.smithi063.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 347 0 --:--:-- --:--:-- --:--:-- 348 2024-05-09T20:10:37.716 INFO:teuthology.orchestra.run.smithi063.stdout:anonymous 2024-05-09T20:10:37.717 INFO:teuthology.run_tasks:Running task tox... 2024-05-09T20:10:37.727 INFO:tasks.tox:Deploying tox from pip... 2024-05-09T20:10:37.728 DEBUG:teuthology.orchestra.run.smithi063:> python3 -m venv /home/ubuntu/cephtest/tox-venv 2024-05-09T20:10:40.742 DEBUG:teuthology.orchestra.run.smithi063:> source /home/ubuntu/cephtest/tox-venv/bin/activate && pip install tox==3.15.0 2024-05-09T20:10:41.455 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting tox==3.15.0 2024-05-09T20:10:41.667 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading tox-3.15.0-py2.py3-none-any.whl (82 kB) 2024-05-09T20:10:41.740 INFO:teuthology.orchestra.run.smithi063.stdout: ---------------------------------------- 82.1/82.1 KB 1.1 MB/s eta 0:00:00 2024-05-09T20:10:41.923 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting pluggy<1,>=0.12.0 2024-05-09T20:10:41.972 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading pluggy-0.13.1-py2.py3-none-any.whl (18 kB) 2024-05-09T20:10:42.091 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting six<2,>=1.14.0 2024-05-09T20:10:42.110 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-09T20:10:42.261 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting filelock<4,>=3.0.0 2024-05-09T20:10:42.282 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading filelock-3.14.0-py3-none-any.whl (12 kB) 2024-05-09T20:10:42.425 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting packaging>=14 2024-05-09T20:10:42.446 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading packaging-24.0-py3-none-any.whl (53 kB) 2024-05-09T20:10:42.476 INFO:teuthology.orchestra.run.smithi063.stdout: ---------------------------------------- 53.5/53.5 KB 1.7 MB/s eta 0:00:00 2024-05-09T20:10:42.580 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting py<2,>=1.4.17 2024-05-09T20:10:42.598 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading py-1.11.0-py2.py3-none-any.whl (98 kB) 2024-05-09T20:10:42.635 INFO:teuthology.orchestra.run.smithi063.stdout: ---------------------------------------- 98.7/98.7 KB 2.8 MB/s eta 0:00:00 2024-05-09T20:10:42.894 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 2024-05-09T20:10:42.915 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading virtualenv-20.26.1-py3-none-any.whl (3.9 MB) 2024-05-09T20:10:43.072 INFO:teuthology.orchestra.run.smithi063.stdout: ---------------------------------------- 3.9/3.9 MB 25.7 MB/s eta 0:00:00 2024-05-09T20:10:43.166 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting toml>=0.9.4 2024-05-09T20:10:43.186 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2024-05-09T20:10:43.369 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting distlib<1,>=0.3.7 2024-05-09T20:10:43.390 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB) 2024-05-09T20:10:43.430 INFO:teuthology.orchestra.run.smithi063.stdout: -------------------------------------- 468.9/468.9 KB 13.1 MB/s eta 0:00:00 2024-05-09T20:10:43.546 INFO:teuthology.orchestra.run.smithi063.stdout:Collecting platformdirs<5,>=3.9.1 2024-05-09T20:10:43.564 INFO:teuthology.orchestra.run.smithi063.stdout: Downloading platformdirs-4.2.1-py3-none-any.whl (17 kB) 2024-05-09T20:10:43.760 INFO:teuthology.orchestra.run.smithi063.stdout:Installing collected packages: distlib, toml, six, py, pluggy, platformdirs, packaging, filelock, virtualenv, tox 2024-05-09T20:10:44.783 INFO:teuthology.orchestra.run.smithi063.stdout:Successfully installed distlib-0.3.8 filelock-3.14.0 packaging-24.0 platformdirs-4.2.1 pluggy-0.13.1 py-1.11.0 six-1.16.0 toml-0.10.2 tox-3.15.0 virtualenv-20.26.1 2024-05-09T20:10:44.870 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T20:10:44.887 INFO:tasks.workunit:Pulling workunits from ref 86872c6b9201173c508ab909aaa65bdb12239dd6 2024-05-09T20:10:44.887 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T20:10:44.888 DEBUG:teuthology.orchestra.run.smithi063:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:10:44.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:10:44.894 INFO:teuthology.orchestra.run.smithi063.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-09T20:10:44.894 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:10:44.942 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-09T20:10:44.942 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-09T20:10:44.990 INFO:tasks.workunit:timeout=3h 2024-05-09T20:10:44.990 INFO:tasks.workunit:cleanup=True 2024-05-09T20:10:44.994 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 86872c6b9201173c508ab909aaa65bdb12239dd6 2024-05-09T20:10:45.039 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T20:11:34.224 INFO:tasks.workunit.client.0.smithi063.stderr:Note: switching to '86872c6b9201173c508ab909aaa65bdb12239dd6'. 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr: git switch -c 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr:Or undo this operation with: 2024-05-09T20:11:34.225 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:34.226 INFO:tasks.workunit.client.0.smithi063.stderr: git switch - 2024-05-09T20:11:34.226 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:34.226 INFO:tasks.workunit.client.0.smithi063.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T20:11:34.226 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:34.226 INFO:tasks.workunit.client.0.smithi063.stderr:HEAD is now at 86872c6b920 d4n: Add more in-depth config descriptions and improve naming 2024-05-09T20:11:34.229 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T20:11:34.278 INFO:tasks.workunit.client.0.smithi063.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T20:11:34.281 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T20:11:34.281 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T20:11:35.114 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T20:11:35.195 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T20:11:35.243 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T20:11:35.245 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T20:11:35.245 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T20:11:35.294 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T20:11:35.298 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:11:35.298 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T20:11:35.345 INFO:tasks.workunit:Running workunits matching rgw/run-bucket-check.sh on client.0... 2024-05-09T20:11:35.346 INFO:tasks.workunit:Running workunit rgw/run-bucket-check.sh... 2024-05-09T20:11:35.347 DEBUG:teuthology.orchestra.run.smithi063:workunit test rgw/run-bucket-check.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-05-09T20:11:35.400 INFO:tasks.workunit.client.0.smithi063.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-bucket-check.sh 2024-05-09T20:11:35.400 INFO:tasks.workunit.client.0.smithi063.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-09T20:11:35.400 INFO:tasks.workunit.client.0.smithi063.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ deactivate nondestructive 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n '' ']' 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n '' ']' 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ hash -r 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n '' ']' 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ unset VIRTUAL_ENV 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-05-09T20:11:38.385 INFO:tasks.workunit.client.0.smithi063.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ export VIRTUAL_ENV 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ export PATH 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n '' ']' 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -z '' ']' 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ _OLD_VIRTUAL_PS1= 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ PS1='(rgw) ' 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ export PS1 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ export VIRTUAL_ENV_PROMPT 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-09T20:11:38.386 INFO:tasks.workunit.client.0.smithi063.stderr:++ hash -r 2024-05-09T20:11:38.387 INFO:tasks.workunit.client.0.smithi063.stderr:+ pip install pip --upgrade 2024-05-09T20:11:38.790 INFO:tasks.workunit.client.0.smithi063.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-05-09T20:11:39.043 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting pip 2024-05-09T20:11:39.154 INFO:tasks.workunit.client.0.smithi063.stdout: Downloading pip-24.0-py3-none-any.whl (2.1 MB) 2024-05-09T20:11:39.307 INFO:tasks.workunit.client.0.smithi063.stdout: ---------------------------------------- 2.1/2.1 MB 14.1 MB/s eta 0:00:00 2024-05-09T20:11:39.383 INFO:tasks.workunit.client.0.smithi063.stdout:Installing collected packages: pip 2024-05-09T20:11:39.383 INFO:tasks.workunit.client.0.smithi063.stdout: Attempting uninstall: pip 2024-05-09T20:11:39.384 INFO:tasks.workunit.client.0.smithi063.stdout: Found existing installation: pip 22.0.2 2024-05-09T20:11:39.690 INFO:tasks.workunit.client.0.smithi063.stdout: Uninstalling pip-22.0.2: 2024-05-09T20:11:39.693 INFO:tasks.workunit.client.0.smithi063.stdout: Successfully uninstalled pip-22.0.2 2024-05-09T20:11:40.684 INFO:tasks.workunit.client.0.smithi063.stdout:Successfully installed pip-24.0 2024-05-09T20:11:40.743 INFO:tasks.workunit.client.0.smithi063.stderr:+ pip install boto3 2024-05-09T20:11:41.875 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting boto3 2024-05-09T20:11:41.992 INFO:tasks.workunit.client.0.smithi063.stdout: Downloading boto3-1.34.101-py3-none-any.whl.metadata (6.6 kB) 2024-05-09T20:11:42.724 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting botocore<1.35.0,>=1.34.101 (from boto3) 2024-05-09T20:11:42.741 INFO:tasks.workunit.client.0.smithi063.stdout: Downloading botocore-1.34.101-py3-none-any.whl.metadata (5.7 kB) 2024-05-09T20:11:42.879 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-05-09T20:11:42.897 INFO:tasks.workunit.client.0.smithi063.stdout: Downloading jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-05-09T20:11:43.061 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-05-09T20:11:43.081 INFO:tasks.workunit.client.0.smithi063.stdout: Downloading s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-05-09T20:11:43.267 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:11:43.286 INFO:tasks.workunit.client.0.smithi063.stdout: Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-05-09T20:11:43.501 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:11:43.519 INFO:tasks.workunit.client.0.smithi063.stdout: Downloading urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-05-09T20:11:43.874 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:11:43.891 INFO:tasks.workunit.client.0.smithi063.stdout: Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-05-09T20:11:43.970 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading boto3-1.34.101-py3-none-any.whl (139 kB) 2024-05-09T20:11:44.047 INFO:tasks.workunit.client.0.smithi063.stdout: ---------------------------------------- 139.3/139.3 kB 1.8 MB/s eta 0:00:00 2024-05-09T20:11:44.067 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading botocore-1.34.101-py3-none-any.whl (12.2 MB) 2024-05-09T20:11:44.353 INFO:tasks.workunit.client.0.smithi063.stdout: ---------------------------------------- 12.2/12.2 MB 44.7 MB/s eta 0:00:00 2024-05-09T20:11:44.373 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-05-09T20:11:44.447 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-05-09T20:11:44.506 INFO:tasks.workunit.client.0.smithi063.stdout: ---------------------------------------- 82.2/82.2 kB 1.3 MB/s eta 0:00:00 2024-05-09T20:11:44.527 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-05-09T20:11:44.581 INFO:tasks.workunit.client.0.smithi063.stdout: ---------------------------------------- 229.9/229.9 kB 4.4 MB/s eta 0:00:00 2024-05-09T20:11:44.599 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-05-09T20:11:44.657 INFO:tasks.workunit.client.0.smithi063.stdout: ---------------------------------------- 121.1/121.1 kB 2.0 MB/s eta 0:00:00 2024-05-09T20:11:44.677 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-09T20:11:44.887 INFO:tasks.workunit.client.0.smithi063.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-05-09T20:11:46.108 INFO:tasks.workunit.client.0.smithi063.stdout:Successfully installed boto3-1.34.101 botocore-1.34.101 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-05-09T20:11:46.652 INFO:tasks.workunit.client.0.smithi063.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_bucket_check.py 2024-05-09T20:11:46.809 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin user create --uid check-tester --display-name "Check Testing" --access-key OJODXSLNX4LUNHQG99PA --secret 3l6ffld34qaymfomuh832j94738aie2x4p2o8h6n 2024-05-09T20:11:46.848 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:46.849 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:46.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.894+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:46.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:46.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 realm 2024-05-09T20:11:46.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:46.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:46.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:46.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:46.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:46.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:46.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:46.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.898+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:46.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.902+0000 7fdd10b04980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:46.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.902+0000 7fdd10b04980 20 searching for the correct realm 2024-05-09T20:11:46.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:46.902+0000 7fdd0e618640 0 -- 172.21.15.63:0/3122108066 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fdd08025040 msgr2=0x7fdd0805c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:47.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.122+0000 7fdd0e618640 0 -- 172.21.15.63:0/3122108066 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fdd08025040 msgr2=0x7fdd0805c750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:47.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:47.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:47.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:47.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:47.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:47.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:47.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:47.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:47.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:47.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:47.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.134+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:47.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:47.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 zone default found 2024-05-09T20:11:47.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 4 Realm: () 2024-05-09T20:11:47.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:47.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:47.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:47.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 zonegroup default 2024-05-09T20:11:47.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:47.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:47.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:47.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:47.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:47.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:47.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 started sync module instance, tier type = 2024-05-09T20:11:47.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.138+0000 7fdd10b04980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:47.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 20 add_watcher() i=0 2024-05-09T20:11:47.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 20 add_watcher() i=1 2024-05-09T20:11:47.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 20 add_watcher() i=2 2024-05-09T20:11:47.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 20 add_watcher() i=3 2024-05-09T20:11:47.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 20 add_watcher() i=4 2024-05-09T20:11:47.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 20 add_watcher() i=5 2024-05-09T20:11:47.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 20 add_watcher() i=6 2024-05-09T20:11:47.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 20 add_watcher() i=7 2024-05-09T20:11:47.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.166+0000 7fdd10b04980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:47.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.170+0000 7fdbc57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:47.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:47.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:47.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 5 note: GC not initialized 2024-05-09T20:11:47.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdbbf7fe640 20 reqs_thread_entry: start 2024-05-09T20:11:47.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:47.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 20 Filter name: none 2024-05-09T20:11:47.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdbbd7fa640 20 reqs_thread_entry: start 2024-05-09T20:11:47.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 10 cache get: name=default.rgw.meta+users.uid+check-tester : miss 2024-05-09T20:11:47.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:47.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:47.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x0 2024-05-09T20:11:47.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 10 adding default.rgw.meta+users.uid+check-tester to cache LRU end 2024-05-09T20:11:47.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : miss 2024-05-09T20:11:47.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.174+0000 7fdd10b04980 20 rados->read ofs=0 len=0 2024-05-09T20:11:47.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.178+0000 7fdd10b04980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:47.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.178+0000 7fdd10b04980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x0 2024-05-09T20:11:47.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.178+0000 7fdd10b04980 10 adding default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-05-09T20:11:47.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.178+0000 7fdd10b04980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-05-09T20:11:47.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.178+0000 7fdd10b04980 10 cache get: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA : hit (negative entry) 2024-05-09T20:11:47.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.178+0000 7fdd10b04980 10 cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-05-09T20:11:47.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.178+0000 7fdd10b04980 10 moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-05-09T20:11:47.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.178+0000 7fdd10b04980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:check-tester, ofs0, ns] 2024-05-09T20:11:47.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.182+0000 7fdbf1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 163208757248 cookie 94892587491696 notifier 4514 bl.length()=593 2024-05-09T20:11:47.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.182+0000 7fdbf1ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+check-tester info.flags=0x17 2024-05-09T20:11:47.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.182+0000 7fdbf1ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+check-tester to cache LRU end 2024-05-09T20:11:47.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.214+0000 7fdd10b04980 10 cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-05-09T20:11:47.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.214+0000 7fdd10b04980 10 moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-05-09T20:11:47.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.214+0000 7fdd10b04980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:OJODXSLNX4LUNHQG99PA, ofs0, ns] 2024-05-09T20:11:47.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.214+0000 7fdbf17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 163208757248 cookie 94892587480928 notifier 4514 bl.length()=186 2024-05-09T20:11:47.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.214+0000 7fdbf17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA info.flags=0x7 2024-05-09T20:11:47.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.214+0000 7fdbf17fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+OJODXSLNX4LUNHQG99PA to cache LRU end 2024-05-09T20:11:47.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.218+0000 7fdbc57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:47.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.226+0000 7fdd10b04980 20 remove_watcher() i=0 2024-05-09T20:11:47.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.226+0000 7fdd10b04980 2 removed watcher, disabling cache 2024-05-09T20:11:47.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.226+0000 7fdd10b04980 20 remove_watcher() i=1 2024-05-09T20:11:47.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.230+0000 7fdd10b04980 20 remove_watcher() i=2 2024-05-09T20:11:47.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.234+0000 7fdd10b04980 20 remove_watcher() i=3 2024-05-09T20:11:47.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.234+0000 7fdd10b04980 20 remove_watcher() i=4 2024-05-09T20:11:47.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.238+0000 7fdd10b04980 20 remove_watcher() i=5 2024-05-09T20:11:47.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.238+0000 7fdd10b04980 20 remove_watcher() i=6 2024-05-09T20:11:47.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:47.242+0000 7fdd10b04980 20 remove_watcher() i=7 2024-05-09T20:11:51.122 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: recalculated bucket check stats are correct 2024-05-09T20:11:51.123 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:51.123 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-05-09T20:11:51.162 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:51.163 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:51.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.210+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 realm 2024-05-09T20:11:51.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:51.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:51.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.214+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.218+0000 7f5eabf66980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:51.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.218+0000 7f5eabf66980 20 searching for the correct realm 2024-05-09T20:11:51.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.218+0000 7f5ea9279640 0 -- 172.21.15.63:0/1782100431 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f5ea4018550 msgr2=0x7f5ea402b460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:51.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.430+0000 7f5ea9279640 0 -- 172.21.15.63:0/1782100431 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f5ea403e950 msgr2=0x7f5ea4029440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:11:51.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:51.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:51.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:51.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:51.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:51.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:51.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:51.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:51.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 zone default found 2024-05-09T20:11:51.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 4 Realm: () 2024-05-09T20:11:51.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:51.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:51.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:51.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 zonegroup default 2024-05-09T20:11:51.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.654+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.658+0000 7f5eabf66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.658+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.658+0000 7f5eabf66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.658+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.658+0000 7f5eabf66980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.658+0000 7f5eabf66980 20 started sync module instance, tier type = 2024-05-09T20:11:51.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.658+0000 7f5eabf66980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:51.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.682+0000 7f5eabf66980 20 add_watcher() i=0 2024-05-09T20:11:51.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.682+0000 7f5eabf66980 20 add_watcher() i=1 2024-05-09T20:11:51.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.682+0000 7f5eabf66980 20 add_watcher() i=2 2024-05-09T20:11:51.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.682+0000 7f5eabf66980 20 add_watcher() i=3 2024-05-09T20:11:51.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.682+0000 7f5eabf66980 20 add_watcher() i=4 2024-05-09T20:11:51.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.682+0000 7f5eabf66980 20 add_watcher() i=5 2024-05-09T20:11:51.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.686+0000 7f5eabf66980 20 add_watcher() i=6 2024-05-09T20:11:51.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.686+0000 7f5eabf66980 20 add_watcher() i=7 2024-05-09T20:11:51.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.686+0000 7f5eabf66980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:51.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.690+0000 7f5d5b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:51.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.690+0000 7f5eabf66980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:51.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.690+0000 7f5eabf66980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:51.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 5 note: GC not initialized 2024-05-09T20:11:51.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5d5a7fc640 20 reqs_thread_entry: start 2024-05-09T20:11:51.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:51.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 20 Filter name: none 2024-05-09T20:11:51.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5ccbfff640 20 reqs_thread_entry: start 2024-05-09T20:11:51.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:11:51.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:51.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:11:51.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:11:51.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:11:51.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:51.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:11:51.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:11:51.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:11:51.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:11:51.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:11:51.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:11:51.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.694+0000 7f5eabf66980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:11:51.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.698+0000 7f5eabf66980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:11:51.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.698+0000 7f5eabf66980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:11:51.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.698+0000 7f5eabf66980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:11:51.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.698+0000 7f5eabf66980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:11:51.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.706+0000 7f5d5b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:51.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.710+0000 7f5eabf66980 20 remove_watcher() i=0 2024-05-09T20:11:51.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.710+0000 7f5eabf66980 2 removed watcher, disabling cache 2024-05-09T20:11:51.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.714+0000 7f5eabf66980 20 remove_watcher() i=1 2024-05-09T20:11:51.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.718+0000 7f5eabf66980 20 remove_watcher() i=2 2024-05-09T20:11:51.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.718+0000 7f5eabf66980 20 remove_watcher() i=3 2024-05-09T20:11:51.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.722+0000 7f5eabf66980 20 remove_watcher() i=4 2024-05-09T20:11:51.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.722+0000 7f5eabf66980 20 remove_watcher() i=5 2024-05-09T20:11:51.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.722+0000 7f5eabf66980 20 remove_watcher() i=6 2024-05-09T20:11:51.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.726+0000 7f5eabf66980 20 remove_watcher() i=7 2024-05-09T20:11:51.740 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-05-09T20:11:51.768 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:51.769 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:51.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.818+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.818+0000 7f415620b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.818+0000 7f415620b980 20 realm 2024-05-09T20:11:51.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.818+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:51.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:51.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:51.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:51.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:51.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:51.822+0000 7f415620b980 20 searching for the correct realm 2024-05-09T20:11:52.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.030+0000 7f415351e640 0 -- 172.21.15.63:0/1591084152 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f4144042030 msgr2=0x7f4144046130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:52.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.038+0000 7f415351e640 0 -- 172.21.15.63:0/1591084152 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f4144084040 msgr2=0x7f414407b220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:11:52.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.258+0000 7f415620b980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:52.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.258+0000 7f415620b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:52.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.258+0000 7f415620b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:52.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.258+0000 7f415620b980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:52.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.258+0000 7f415620b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:52.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:52.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:52.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 zone default found 2024-05-09T20:11:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 4 Realm: () 2024-05-09T20:11:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 zonegroup default 2024-05-09T20:11:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.262+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.266+0000 7f415620b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.266+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.266+0000 7f415620b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.266+0000 7f415620b980 20 started sync module instance, tier type = 2024-05-09T20:11:52.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.266+0000 7f415620b980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:52.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.290+0000 7f415620b980 20 add_watcher() i=0 2024-05-09T20:11:52.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.290+0000 7f415620b980 20 add_watcher() i=1 2024-05-09T20:11:52.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.290+0000 7f415620b980 20 add_watcher() i=2 2024-05-09T20:11:52.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.290+0000 7f415620b980 20 add_watcher() i=3 2024-05-09T20:11:52.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.290+0000 7f415620b980 20 add_watcher() i=4 2024-05-09T20:11:52.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.290+0000 7f415620b980 20 add_watcher() i=5 2024-05-09T20:11:52.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.290+0000 7f415620b980 20 add_watcher() i=6 2024-05-09T20:11:52.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.294+0000 7f415620b980 20 add_watcher() i=7 2024-05-09T20:11:52.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.294+0000 7f415620b980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:52.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f4009fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:52.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f415620b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:52.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f415620b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:52.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f415620b980 5 note: GC not initialized 2024-05-09T20:11:52.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f4008fe1640 20 reqs_thread_entry: start 2024-05-09T20:11:52.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f415620b980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:52.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f415620b980 20 Filter name: none 2024-05-09T20:11:52.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f3f827fc640 20 reqs_thread_entry: start 2024-05-09T20:11:52.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.298+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:52.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:52.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:11:52.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:52.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:52.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.302+0000 7f415620b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:11:52.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.306+0000 7f4009fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:52.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.310+0000 7f415620b980 20 remove_watcher() i=0 2024-05-09T20:11:52.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.310+0000 7f415620b980 2 removed watcher, disabling cache 2024-05-09T20:11:52.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.314+0000 7f415620b980 20 remove_watcher() i=1 2024-05-09T20:11:52.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.318+0000 7f415620b980 20 remove_watcher() i=2 2024-05-09T20:11:52.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.318+0000 7f415620b980 20 remove_watcher() i=3 2024-05-09T20:11:52.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.322+0000 7f415620b980 20 remove_watcher() i=4 2024-05-09T20:11:52.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.322+0000 7f415620b980 20 remove_watcher() i=5 2024-05-09T20:11:52.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.326+0000 7f415620b980 20 remove_watcher() i=6 2024-05-09T20:11:52.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.326+0000 7f415620b980 20 remove_watcher() i=7 2024-05-09T20:11:52.337 INFO:tasks.workunit.client.0.smithi063.stderr:{'rgw.main': {'size': 54, 'size_actual': 24576, 'size_utilized': 111, 'size_kb': 1, 'size_kb_actual': 24, 'size_kb_utilized': 1, 'num_objects': 6}} 2024-05-09T20:11:52.337 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check unlinked does not report normal entries 2024-05-09T20:11:52.337 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:52.337 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-05-09T20:11:52.374 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:52.374 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:52.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.422+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.419 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.422+0000 7fca82f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.419 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.422+0000 7fca82f62980 20 realm 2024-05-09T20:11:52.419 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.422+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.422+0000 7fca82f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.422+0000 7fca82f62980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:52.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.422+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.426+0000 7fca82f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.426+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.426+0000 7fca82f62980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:52.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.426+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.426+0000 7fca82f62980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:52.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.426+0000 7fca82f62980 20 searching for the correct realm 2024-05-09T20:11:52.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.430+0000 7fca7bfff640 0 -- 172.21.15.63:0/1186035701 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fca74027040 msgr2=0x7fca74031450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:52.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.642+0000 7fca7bfff640 0 -- 172.21.15.63:0/1186035701 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7fca74008050 msgr2=0x7fca7404e410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:52.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.866+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:52.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 zone default found 2024-05-09T20:11:52.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 4 Realm: () 2024-05-09T20:11:52.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:52.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:52.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:52.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 zonegroup default 2024-05-09T20:11:52.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:52.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 started sync module instance, tier type = 2024-05-09T20:11:52.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.870+0000 7fca82f62980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:52.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 20 add_watcher() i=0 2024-05-09T20:11:52.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 20 add_watcher() i=1 2024-05-09T20:11:52.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 20 add_watcher() i=2 2024-05-09T20:11:52.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 20 add_watcher() i=3 2024-05-09T20:11:52.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 20 add_watcher() i=4 2024-05-09T20:11:52.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 20 add_watcher() i=5 2024-05-09T20:11:52.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 20 add_watcher() i=6 2024-05-09T20:11:52.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 20 add_watcher() i=7 2024-05-09T20:11:52.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.898+0000 7fca82f62980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:52.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.902+0000 7fc9377e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:52.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:52.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:52.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 5 note: GC not initialized 2024-05-09T20:11:52.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fc9367e4640 20 reqs_thread_entry: start 2024-05-09T20:11:52.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:52.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 Filter name: none 2024-05-09T20:11:52.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fc92ffff640 20 reqs_thread_entry: start 2024-05-09T20:11:52.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:11:52.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:52.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:11:52.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:11:52.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:11:52.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 rados->read ofs=0 len=0 2024-05-09T20:11:52.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:52.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:11:52.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:11:52.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:11:52.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:11:52.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:11:52.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.906+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:11:52.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.910+0000 7fca82f62980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-09T20:11:52.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.910+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:52.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.910+0000 7fca82f62980 1 NOTICE: finished shard 1 (0 entries found) 2024-05-09T20:11:52.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.910+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:11:52.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.910+0000 7fca82f62980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-09T20:11:52.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.910+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:11:52.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.910+0000 7fca82f62980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-09T20:11:52.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.910+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:11:52.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-09T20:11:52.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:52.909 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 1 NOTICE: finished shard 5 (0 entries found) 2024-05-09T20:11:52.909 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:11:52.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-09T20:11:52.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:11:52.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-09T20:11:52.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:52.911 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 1 NOTICE: finished shard 8 (0 entries found) 2024-05-09T20:11:52.911 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.914+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:11:52.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.918+0000 7fca82f62980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-09T20:11:52.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.918+0000 7fca82f62980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:52.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.918+0000 7fca82f62980 1 NOTICE: finished shard 10 (0 entries found) 2024-05-09T20:11:52.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.918+0000 7fca82f62980 1 NOTICE: finished all shards (0 entries found) 2024-05-09T20:11:52.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.918+0000 7fc9377e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:52.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.926+0000 7fca82f62980 20 remove_watcher() i=0 2024-05-09T20:11:52.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.926+0000 7fca82f62980 2 removed watcher, disabling cache 2024-05-09T20:11:52.922 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.926+0000 7fca82f62980 20 remove_watcher() i=1 2024-05-09T20:11:52.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.930+0000 7fca82f62980 20 remove_watcher() i=2 2024-05-09T20:11:52.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.930+0000 7fca82f62980 20 remove_watcher() i=3 2024-05-09T20:11:52.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.934+0000 7fca82f62980 20 remove_watcher() i=4 2024-05-09T20:11:52.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.934+0000 7fca82f62980 20 remove_watcher() i=5 2024-05-09T20:11:52.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.938+0000 7fca82f62980 20 remove_watcher() i=6 2024-05-09T20:11:52.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:52.938+0000 7fca82f62980 20 remove_watcher() i=7 2024-05-09T20:11:52.954 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-05-09T20:11:53.386 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-05-09T20:11:54.989 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:11:54.990 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-05-09T20:11:55.026 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:55.026 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:55.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.074+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.074+0000 7fb662e73980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.074+0000 7fb662e73980 20 realm 2024-05-09T20:11:55.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.074+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:55.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.073 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:55.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:55.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.078+0000 7fb662e73980 20 searching for the correct realm 2024-05-09T20:11:55.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.082+0000 7fb660987640 0 -- 172.21.15.63:0/4173744662 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fb658124a60 msgr2=0x7fb65810e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:55.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.298+0000 7fb660987640 0 -- 172.21.15.63:0/4173744662 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fb658124a60 msgr2=0x7fb65810e150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:55.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.310+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:55.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 zone default found 2024-05-09T20:11:55.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 4 Realm: () 2024-05-09T20:11:55.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:55.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:55.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:55.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 zonegroup default 2024-05-09T20:11:55.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 started sync module instance, tier type = 2024-05-09T20:11:55.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.314+0000 7fb662e73980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:55.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.346+0000 7fb662e73980 20 add_watcher() i=0 2024-05-09T20:11:55.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.346+0000 7fb662e73980 20 add_watcher() i=1 2024-05-09T20:11:55.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.346+0000 7fb662e73980 20 add_watcher() i=2 2024-05-09T20:11:55.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.346+0000 7fb662e73980 20 add_watcher() i=3 2024-05-09T20:11:55.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.346+0000 7fb662e73980 20 add_watcher() i=4 2024-05-09T20:11:55.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.346+0000 7fb662e73980 20 add_watcher() i=5 2024-05-09T20:11:55.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.346+0000 7fb662e73980 20 add_watcher() i=6 2024-05-09T20:11:55.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.350+0000 7fb662e73980 20 add_watcher() i=7 2024-05-09T20:11:55.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.350+0000 7fb662e73980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:55.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb5177e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:55.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:55.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:55.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 5 note: GC not initialized 2024-05-09T20:11:55.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb5167e4640 20 reqs_thread_entry: start 2024-05-09T20:11:55.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:55.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 Filter name: none 2024-05-09T20:11:55.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb50f7fe640 20 reqs_thread_entry: start 2024-05-09T20:11:55.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:55.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:55.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:11:55.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:11:55.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 INFO: main: starting shard=0 2024-05-09T20:11:55.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.354+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:11:55.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 INFO: main: starting shard=1 2024-05-09T20:11:55.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:55.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 INFO: main: starting shard=2 2024-05-09T20:11:55.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:11:55.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 INFO: main: starting shard=3 2024-05-09T20:11:55.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.358+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:11:55.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 INFO: main: starting shard=4 2024-05-09T20:11:55.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:11:55.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 INFO: main: starting shard=5 2024-05-09T20:11:55.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:55.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 INFO: main: starting shard=6 2024-05-09T20:11:55.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.362+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:11:55.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: starting shard=7 2024-05-09T20:11:55.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:11:55.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: starting shard=8 2024-05-09T20:11:55.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:55.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-05-09T20:11:55.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: starting shard=9 2024-05-09T20:11:55.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:11:55.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: starting shard=10 2024-05-09T20:11:55.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:55.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:55.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.366+0000 7fb662e73980 20 INFO: main: done 2024-05-09T20:11:55.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.370+0000 7fb5177e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:55.370 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.374+0000 7fb662e73980 20 remove_watcher() i=0 2024-05-09T20:11:55.371 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.374+0000 7fb662e73980 2 removed watcher, disabling cache 2024-05-09T20:11:55.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.378+0000 7fb662e73980 20 remove_watcher() i=1 2024-05-09T20:11:55.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.378+0000 7fb662e73980 20 remove_watcher() i=2 2024-05-09T20:11:55.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.382+0000 7fb662e73980 20 remove_watcher() i=3 2024-05-09T20:11:55.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.386+0000 7fb662e73980 20 remove_watcher() i=4 2024-05-09T20:11:55.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.386+0000 7fb662e73980 20 remove_watcher() i=5 2024-05-09T20:11:55.384 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.390+0000 7fb662e73980 20 remove_watcher() i=6 2024-05-09T20:11:55.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.390+0000 7fb662e73980 20 remove_watcher() i=7 2024-05-09T20:11:55.641 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:11:55.641 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-05-09T20:11:55.678 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:55.678 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:55.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.726+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 realm 2024-05-09T20:11:55.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:55.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:55.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:55.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.730+0000 7fe7b7f90980 20 searching for the correct realm 2024-05-09T20:11:55.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.734+0000 7fe7b62a5640 0 -- 172.21.15.63:0/2104389266 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fe7a00b3400 msgr2=0x7fe7a00b3820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:11:55.943 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.946+0000 7fe7b52a3640 0 -- 172.21.15.63:0/2104389266 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fe7a8008040 msgr2=0x7fe7a80769b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:55.958 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:55.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:55.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:55.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:55.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:55.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:55.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.962+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:55.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:55.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 zone default found 2024-05-09T20:11:55.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 4 Realm: () 2024-05-09T20:11:55.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:55.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:55.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:55.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 zonegroup default 2024-05-09T20:11:55.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:55.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:55.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 started sync module instance, tier type = 2024-05-09T20:11:55.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.966+0000 7fe7b7f90980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:55.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 20 add_watcher() i=0 2024-05-09T20:11:55.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 20 add_watcher() i=1 2024-05-09T20:11:55.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 20 add_watcher() i=2 2024-05-09T20:11:55.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 20 add_watcher() i=3 2024-05-09T20:11:55.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 20 add_watcher() i=4 2024-05-09T20:11:55.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 20 add_watcher() i=5 2024-05-09T20:11:55.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 20 add_watcher() i=6 2024-05-09T20:11:55.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 20 add_watcher() i=7 2024-05-09T20:11:55.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:55.998+0000 7fe7b7f90980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:55.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.002+0000 7fe6677fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:56.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:56.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:56.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 5 note: GC not initialized 2024-05-09T20:11:56.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe6667fc640 20 reqs_thread_entry: start 2024-05-09T20:11:56.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:56.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 Filter name: none 2024-05-09T20:11:56.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe5d7fff640 20 reqs_thread_entry: start 2024-05-09T20:11:56.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:56.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:56.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:11:56.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:11:56.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 INFO: main: starting shard=0 2024-05-09T20:11:56.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.006+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:11:56.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 INFO: main: starting shard=1 2024-05-09T20:11:56.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:56.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 INFO: main: starting shard=2 2024-05-09T20:11:56.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:11:56.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 INFO: main: starting shard=3 2024-05-09T20:11:56.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:11:56.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 INFO: main: starting shard=4 2024-05-09T20:11:56.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.010+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:11:56.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: starting shard=5 2024-05-09T20:11:56.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:56.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-05-09T20:11:56.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: starting shard=6 2024-05-09T20:11:56.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:11:56.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: starting shard=7 2024-05-09T20:11:56.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:11:56.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: starting shard=8 2024-05-09T20:11:56.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:56.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 INFO: main: starting shard=9 2024-05-09T20:11:56.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.014+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:11:56.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.018+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.018+0000 7fe7b7f90980 20 INFO: main: starting shard=10 2024-05-09T20:11:56.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.018+0000 7fe7b7f90980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:56.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.018+0000 7fe7b7f90980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.018+0000 7fe7b7f90980 20 INFO: main: done 2024-05-09T20:11:56.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.018+0000 7fe6677fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:56.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.026+0000 7fe7b7f90980 20 remove_watcher() i=0 2024-05-09T20:11:56.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.026+0000 7fe7b7f90980 2 removed watcher, disabling cache 2024-05-09T20:11:56.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.026+0000 7fe7b7f90980 20 remove_watcher() i=1 2024-05-09T20:11:56.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.030+0000 7fe7b7f90980 20 remove_watcher() i=2 2024-05-09T20:11:56.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.034+0000 7fe7b7f90980 20 remove_watcher() i=3 2024-05-09T20:11:56.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.034+0000 7fe7b7f90980 20 remove_watcher() i=4 2024-05-09T20:11:56.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.038+0000 7fe7b7f90980 20 remove_watcher() i=5 2024-05-09T20:11:56.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.042+0000 7fe7b7f90980 20 remove_watcher() i=6 2024-05-09T20:11:56.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.042+0000 7fe7b7f90980 20 remove_watcher() i=7 2024-05-09T20:11:56.289 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:11:56.289 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-05-09T20:11:56.326 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:56.327 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:56.371 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.374+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 realm 2024-05-09T20:11:56.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:56.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:56.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.378+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.382+0000 7f9585a20980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:56.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.382+0000 7f9585a20980 20 searching for the correct realm 2024-05-09T20:11:56.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.382+0000 7f9583534640 0 -- 172.21.15.63:0/3600027106 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f957c0df040 msgr2=0x7f957c0df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:11:56.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.598+0000 7f9583534640 0 -- 172.21.15.63:0/3600027106 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f957c0df040 msgr2=0x7f957c0df420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:56.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.610+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 zone default found 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 4 Realm: () 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 zonegroup default 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 started sync module instance, tier type = 2024-05-09T20:11:56.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.614+0000 7f9585a20980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:56.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.642+0000 7f9585a20980 20 add_watcher() i=0 2024-05-09T20:11:56.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.642+0000 7f9585a20980 20 add_watcher() i=1 2024-05-09T20:11:56.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.642+0000 7f9585a20980 20 add_watcher() i=2 2024-05-09T20:11:56.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.642+0000 7f9585a20980 20 add_watcher() i=3 2024-05-09T20:11:56.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.642+0000 7f9585a20980 20 add_watcher() i=4 2024-05-09T20:11:56.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.642+0000 7f9585a20980 20 add_watcher() i=5 2024-05-09T20:11:56.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.642+0000 7f9585a20980 20 add_watcher() i=6 2024-05-09T20:11:56.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.646+0000 7f9585a20980 20 add_watcher() i=7 2024-05-09T20:11:56.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.646+0000 7f9585a20980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:56.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.650+0000 7f9439fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:56.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.650+0000 7f9585a20980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:56.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.650+0000 7f9585a20980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:56.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.650+0000 7f9585a20980 5 note: GC not initialized 2024-05-09T20:11:56.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.650+0000 7f9438fe1640 20 reqs_thread_entry: start 2024-05-09T20:11:56.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:56.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 Filter name: none 2024-05-09T20:11:56.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f93b27fc640 20 reqs_thread_entry: start 2024-05-09T20:11:56.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:56.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:56.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:11:56.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:11:56.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 INFO: main: starting shard=0 2024-05-09T20:11:56.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.654+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:11:56.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 INFO: main: starting shard=1 2024-05-09T20:11:56.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:56.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-09T20:11:56.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 INFO: main: starting shard=2 2024-05-09T20:11:56.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:11:56.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 INFO: main: starting shard=3 2024-05-09T20:11:56.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:11:56.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 INFO: main: starting shard=4 2024-05-09T20:11:56.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.658+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:11:56.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 INFO: main: starting shard=5 2024-05-09T20:11:56.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:56.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 INFO: main: starting shard=6 2024-05-09T20:11:56.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:11:56.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 INFO: main: starting shard=7 2024-05-09T20:11:56.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:11:56.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 INFO: main: starting shard=8 2024-05-09T20:11:56.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.662+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:56.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9585a20980 20 INFO: main: starting shard=9 2024-05-09T20:11:56.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:11:56.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9585a20980 20 INFO: main: starting shard=10 2024-05-09T20:11:56.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9585a20980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:56.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9585a20980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:56.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9585a20980 20 INFO: main: done 2024-05-09T20:11:56.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.666+0000 7f9439fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:56.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.674+0000 7f9585a20980 20 remove_watcher() i=0 2024-05-09T20:11:56.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.674+0000 7f9585a20980 2 removed watcher, disabling cache 2024-05-09T20:11:56.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.674+0000 7f9585a20980 20 remove_watcher() i=1 2024-05-09T20:11:56.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.678+0000 7f9585a20980 20 remove_watcher() i=2 2024-05-09T20:11:56.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.678+0000 7f9585a20980 20 remove_watcher() i=3 2024-05-09T20:11:56.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.682+0000 7f9585a20980 20 remove_watcher() i=4 2024-05-09T20:11:56.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.686+0000 7f9585a20980 20 remove_watcher() i=5 2024-05-09T20:11:56.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.686+0000 7f9585a20980 20 remove_watcher() i=6 2024-05-09T20:11:56.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.690+0000 7f9585a20980 20 remove_watcher() i=7 2024-05-09T20:11:56.853 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:11:56.853 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-05-09T20:11:56.890 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:56.890 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:56.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.938+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.938+0000 7f0517a46980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.938+0000 7f0517a46980 20 realm 2024-05-09T20:11:56.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.938+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:56.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:56.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:56.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:56.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:56.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.942+0000 7f0517a46980 20 searching for the correct realm 2024-05-09T20:11:56.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:56.946+0000 7f0515d5b640 0 -- 172.21.15.63:0/1035171282 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f050c032060 msgr2=0x7f050c05b8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:11:57.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.158+0000 7f0514d59640 0 -- 172.21.15.63:0/1035171282 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f0508008050 msgr2=0x7f05080870b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:57.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.174+0000 7f0517a46980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:57.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.174+0000 7f0517a46980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:57.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.174+0000 7f0517a46980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:57.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.174+0000 7f0517a46980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:57.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.174+0000 7f0517a46980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:57.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.174+0000 7f0517a46980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:57.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.174+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:57.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:57.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:57.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:57.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:57.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:57.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 zone default found 2024-05-09T20:11:57.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 4 Realm: () 2024-05-09T20:11:57.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:57.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:57.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:57.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 zonegroup default 2024-05-09T20:11:57.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:57.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:57.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:57.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:57.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.178+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:57.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.182+0000 7f0517a46980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:57.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.182+0000 7f0517a46980 20 started sync module instance, tier type = 2024-05-09T20:11:57.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.182+0000 7f0517a46980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:57.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 20 add_watcher() i=0 2024-05-09T20:11:57.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 20 add_watcher() i=1 2024-05-09T20:11:57.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 20 add_watcher() i=2 2024-05-09T20:11:57.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 20 add_watcher() i=3 2024-05-09T20:11:57.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 20 add_watcher() i=4 2024-05-09T20:11:57.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 20 add_watcher() i=5 2024-05-09T20:11:57.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 20 add_watcher() i=6 2024-05-09T20:11:57.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 20 add_watcher() i=7 2024-05-09T20:11:57.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.206+0000 7f0517a46980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:57.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.210+0000 7f03cb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:57.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.210+0000 7f0517a46980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:57.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.210+0000 7f0517a46980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:57.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.210+0000 7f0517a46980 5 note: GC not initialized 2024-05-09T20:11:57.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f03ca7e4640 20 reqs_thread_entry: start 2024-05-09T20:11:57.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:57.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 Filter name: none 2024-05-09T20:11:57.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f033bfff640 20 reqs_thread_entry: start 2024-05-09T20:11:57.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:57.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:57.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 rados->read ofs=0 len=0 2024-05-09T20:11:57.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:57.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:11:57.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:11:57.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 INFO: main: starting shard=0 2024-05-09T20:11:57.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:11:57.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 INFO: main: starting shard=1 2024-05-09T20:11:57.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.214+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:57.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 INFO: main: starting shard=2 2024-05-09T20:11:57.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:11:57.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 INFO: main: starting shard=3 2024-05-09T20:11:57.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:11:57.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 INFO: main: starting shard=4 2024-05-09T20:11:57.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:11:57.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 INFO: main: starting shard=5 2024-05-09T20:11:57.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.218+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:57.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 INFO: main: starting shard=6 2024-05-09T20:11:57.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:11:57.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 INFO: main: starting shard=7 2024-05-09T20:11:57.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:11:57.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 INFO: main: starting shard=8 2024-05-09T20:11:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 INFO: main: starting shard=9 2024-05-09T20:11:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.222+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:11:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.226+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:11:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.226+0000 7f0517a46980 20 INFO: main: starting shard=10 2024-05-09T20:11:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.226+0000 7f0517a46980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.226+0000 7f0517a46980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-09T20:11:57.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.226+0000 7f0517a46980 20 INFO: main: done 2024-05-09T20:11:57.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.226+0000 7f03cb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:57.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.234+0000 7f0517a46980 20 remove_watcher() i=0 2024-05-09T20:11:57.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.234+0000 7f0517a46980 2 removed watcher, disabling cache 2024-05-09T20:11:57.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.234+0000 7f0517a46980 20 remove_watcher() i=1 2024-05-09T20:11:57.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.238+0000 7f0517a46980 20 remove_watcher() i=2 2024-05-09T20:11:57.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.242+0000 7f0517a46980 20 remove_watcher() i=3 2024-05-09T20:11:57.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.242+0000 7f0517a46980 20 remove_watcher() i=4 2024-05-09T20:11:57.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.246+0000 7f0517a46980 20 remove_watcher() i=5 2024-05-09T20:11:57.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.246+0000 7f0517a46980 20 remove_watcher() i=6 2024-05-09T20:11:57.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:57.250+0000 7f0517a46980 20 remove_watcher() i=7 2024-05-09T20:11:57.290 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-05-09T20:11:57.717 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-05-09T20:11:58.149 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check unlinked finds unlistable entries 2024-05-09T20:11:58.150 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:58.150 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-05-09T20:11:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:58.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:58.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 20 realm 2024-05-09T20:11:58.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:58.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:58.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:58.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.234+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.238+0000 7fbf04081980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:58.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.238+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.238+0000 7fbf04081980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:58.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.238+0000 7fbf04081980 20 searching for the correct realm 2024-05-09T20:11:58.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.242+0000 7fbf01b95640 0 -- 172.21.15.63:0/2501030439 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fbefc119af0 msgr2=0x7fbefc0f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:11:58.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.458+0000 7fbf01b95640 0 -- 172.21.15.63:0/2501030439 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fbefc119af0 msgr2=0x7fbefc0f2830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:58.465 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:58.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:58.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:58.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:58.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:58.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:58.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:58.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:58.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.470+0000 7fbf04081980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:58.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 zone default found 2024-05-09T20:11:58.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 4 Realm: () 2024-05-09T20:11:58.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:58.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:58.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:58.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 zonegroup default 2024-05-09T20:11:58.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:58.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:58.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:58.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 started sync module instance, tier type = 2024-05-09T20:11:58.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.474+0000 7fbf04081980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:58.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.502+0000 7fbf04081980 20 add_watcher() i=0 2024-05-09T20:11:58.499 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.502+0000 7fbf04081980 20 add_watcher() i=1 2024-05-09T20:11:58.499 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.502+0000 7fbf04081980 20 add_watcher() i=2 2024-05-09T20:11:58.499 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.502+0000 7fbf04081980 20 add_watcher() i=3 2024-05-09T20:11:58.499 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.502+0000 7fbf04081980 20 add_watcher() i=4 2024-05-09T20:11:58.499 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.502+0000 7fbf04081980 20 add_watcher() i=5 2024-05-09T20:11:58.499 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.502+0000 7fbf04081980 20 add_watcher() i=6 2024-05-09T20:11:58.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.506+0000 7fbf04081980 20 add_watcher() i=7 2024-05-09T20:11:58.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.506+0000 7fbf04081980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:58.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbdb37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:58.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbf04081980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:58.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbf04081980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:58.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbf04081980 5 note: GC not initialized 2024-05-09T20:11:58.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbdb27fc640 20 reqs_thread_entry: start 2024-05-09T20:11:58.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbf04081980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:58.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbf04081980 20 Filter name: none 2024-05-09T20:11:58.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbd23fff640 20 reqs_thread_entry: start 2024-05-09T20:11:58.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbf04081980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:11:58.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.510+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:58.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:11:58.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:11:58.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:11:58.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 20 rados->read ofs=0 len=0 2024-05-09T20:11:58.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:58.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:11:58.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:11:58.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:11:58.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:11:58.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:11:58.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:11:58.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-09T20:11:58.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:58.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.514+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:11:58.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.518+0000 7fbf04081980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-09T20:11:58.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.518+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:11:58.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.518+0000 7fbf04081980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-09T20:11:58.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.518+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:11:58.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.518+0000 7fbf04081980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-09T20:11:58.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.518+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:58.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:11:58.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-09T20:11:58.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:11:58.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-09T20:11:58.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:58.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:11:58.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-09T20:11:58.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:58.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 1 NOTICE: finished shard 1 (1 entries found) 2024-05-09T20:11:58.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 1 NOTICE: finished shard 5 (1 entries found) 2024-05-09T20:11:58.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.522+0000 7fbf04081980 1 NOTICE: finished shard 8 (1 entries found) 2024-05-09T20:11:58.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.526+0000 7fbf04081980 1 NOTICE: finished shard 10 (1 entries found) 2024-05-09T20:11:58.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.526+0000 7fbf04081980 1 NOTICE: finished all shards (4 entries found) 2024-05-09T20:11:58.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.526+0000 7fbdb37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:58.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.530+0000 7fbf04081980 20 remove_watcher() i=0 2024-05-09T20:11:58.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.530+0000 7fbf04081980 2 removed watcher, disabling cache 2024-05-09T20:11:58.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.534+0000 7fbf04081980 20 remove_watcher() i=1 2024-05-09T20:11:58.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.538+0000 7fbf04081980 20 remove_watcher() i=2 2024-05-09T20:11:58.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.538+0000 7fbf04081980 20 remove_watcher() i=3 2024-05-09T20:11:58.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.542+0000 7fbf04081980 20 remove_watcher() i=4 2024-05-09T20:11:58.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.542+0000 7fbf04081980 20 remove_watcher() i=5 2024-05-09T20:11:58.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.546+0000 7fbf04081980 20 remove_watcher() i=6 2024-05-09T20:11:58.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:58.546+0000 7fbf04081980 20 remove_watcher() i=7 2024-05-09T20:11:58.565 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: unlinked entries are not listable 2024-05-09T20:11:58.565 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:58.724 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: GET returns 404 for unlinked entry keys that have no other versions 2024-05-09T20:11:58.725 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:58.961 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check unlinked fixes unlistable entries 2024-05-09T20:11:58.961 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:11:58.961 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-05-09T20:11:59.002 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:11:59.003 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:11:59.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.050+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:59.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 realm 2024-05-09T20:11:59.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:59.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:11:59.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:59.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:59.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.054+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.052 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.058+0000 7ff7ae689980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:11:59.052 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.058+0000 7ff7ae689980 20 searching for the correct realm 2024-05-09T20:11:59.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.062+0000 7ff7ac19d640 0 -- 172.21.15.63:0/2674409565 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7ff7a405cc00 msgr2=0x7ff7a405af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:59.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.278+0000 7ff7ac19d640 0 -- 172.21.15.63:0/2674409565 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7ff7a405cc00 msgr2=0x7ff7a405af60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:11:59.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:11:59.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:11:59.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:11:59.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:11:59.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:11:59.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:11:59.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:59.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:11:59.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.290+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:11:59.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 zone default found 2024-05-09T20:11:59.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 4 Realm: () 2024-05-09T20:11:59.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:11:59.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:11:59.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:11:59.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 zonegroup default 2024-05-09T20:11:59.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:59.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:59.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:11:59.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 started sync module instance, tier type = 2024-05-09T20:11:59.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.294+0000 7ff7ae689980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:11:59.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.318+0000 7ff7ae689980 20 add_watcher() i=0 2024-05-09T20:11:59.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.318+0000 7ff7ae689980 20 add_watcher() i=1 2024-05-09T20:11:59.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.318+0000 7ff7ae689980 20 add_watcher() i=2 2024-05-09T20:11:59.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.318+0000 7ff7ae689980 20 add_watcher() i=3 2024-05-09T20:11:59.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.318+0000 7ff7ae689980 20 add_watcher() i=4 2024-05-09T20:11:59.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.318+0000 7ff7ae689980 20 add_watcher() i=5 2024-05-09T20:11:59.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.322+0000 7ff7ae689980 20 add_watcher() i=6 2024-05-09T20:11:59.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.322+0000 7ff7ae689980 20 add_watcher() i=7 2024-05-09T20:11:59.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.322+0000 7ff7ae689980 2 all 8 watchers are set, enabling cache 2024-05-09T20:11:59.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff662fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:59.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:11:59.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:11:59.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 5 note: GC not initialized 2024-05-09T20:11:59.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff661fdb640 20 reqs_thread_entry: start 2024-05-09T20:11:59.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 20 init_complete bucket index max shards: 11 2024-05-09T20:11:59.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 20 Filter name: none 2024-05-09T20:11:59.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff5d37fe640 20 reqs_thread_entry: start 2024-05-09T20:11:59.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:11:59.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:11:59.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:11:59.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:11:59.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:11:59.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.326+0000 7ff7ae689980 20 rados->read ofs=0 len=0 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 1 NOTICE: finished shard 0 (0 entries removed) 2024-05-09T20:11:59.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:59.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:11:59.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 1 NOTICE: finished shard 2 (0 entries removed) 2024-05-09T20:11:59.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:11:59.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 1 NOTICE: finished shard 3 (0 entries removed) 2024-05-09T20:11:59.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.330+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:11:59.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.334+0000 7ff7ae689980 1 NOTICE: finished shard 4 (0 entries removed) 2024-05-09T20:11:59.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.334+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:59.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.334+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:11:59.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.334+0000 7ff7ae689980 1 NOTICE: finished shard 6 (0 entries removed) 2024-05-09T20:11:59.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.334+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:11:59.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.334+0000 7ff7ae689980 1 NOTICE: finished shard 7 (0 entries removed) 2024-05-09T20:11:59.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.334+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:59.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.334+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:11:59.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 1 NOTICE: finished shard 9 (0 entries removed) 2024-05-09T20:11:59.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:59.333 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:59.333 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e528af60 obj=check-bucket:e state=0x55c4e5539888 s->prefetch_data=0 2024-05-09T20:11:59.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:59.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e523a290 obj=check-bucket:d state=0x55c4e553e988 s->prefetch_data=0 2024-05-09T20:11:59.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:59.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e55433c0 obj=check-bucket:c state=0x55c4e553a3e8 s->prefetch_data=0 2024-05-09T20:11:59.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:59.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e5544770 obj=check-bucket:f state=0x55c4e553af48 s->prefetch_data=0 2024-05-09T20:11:59.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state: setting s->obj_tag to 7afuffu6qerqpmvjchva2nhexsd2opto 2024-05-09T20:11:59.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state_impl: setting s->olh_tag to a2q7k212tjvhfcka067g5xxuuh2jc37o 2024-05-09T20:11:59.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=a2q7k212tjvhfcka067g5xxuuh2jc37o 2024-05-09T20:11:59.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state: setting s->obj_tag to c47i492hokhl3ozh78b1w4pt06k2wavw 2024-05-09T20:11:59.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state_impl: setting s->olh_tag to x6r4t0f071lcr3ofcquyb57cxnz3k4ss 2024-05-09T20:11:59.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=x6r4t0f071lcr3ofcquyb57cxnz3k4ss 2024-05-09T20:11:59.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state: setting s->obj_tag to xpvisso0krd5y2i368evh4ewrljknr3r 2024-05-09T20:11:59.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 get_obj_state_impl: setting s->olh_tag to tp6l7gk1e302gcdwlxwis3qc3u4ftbtk 2024-05-09T20:11:59.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.338+0000 7ff7ae689980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=tp6l7gk1e302gcdwlxwis3qc3u4ftbtk 2024-05-09T20:11:59.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.342+0000 7ff7ae689980 20 get_obj_state: setting s->obj_tag to md7fbr0e36xzhk8xwhlw0kkyyuh90l1h 2024-05-09T20:11:59.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.342+0000 7ff7ae689980 20 get_obj_state_impl: setting s->olh_tag to 3ptohk6b9t6892whi9q475rspodp3yln 2024-05-09T20:11:59.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.342+0000 7ff7ae689980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3ptohk6b9t6892whi9q475rspodp3yln 2024-05-09T20:11:59.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.342+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:59.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.346+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:59.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.346+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:59.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.350+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:59.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.350+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:59.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.350+0000 7ff7ae689980 20 olh_log_entry: epoch=3 op=3 key=d[AXUzmGZFMkuZrYaKvH7t2aVaYHOlzQm] 2024-05-09T20:11:59.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.350+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e523a290 obj=check-bucket:_:AXUzmGZFMkuZrYaKvH7t2aVaYHOlzQm_d state=0x55c4e55e54d8 s->prefetch_data=0 2024-05-09T20:11:59.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.354+0000 7ff7ae689980 10 manifest: total_size = 11 2024-05-09T20:11:59.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.354+0000 7ff7ae689980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6046464290849141159 2024-05-09T20:11:59.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.354+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e523a290 obj=check-bucket:_:AXUzmGZFMkuZrYaKvH7t2aVaYHOlzQm_d state=0x55c4e55e54d8 s->prefetch_data=0 2024-05-09T20:11:59.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.354+0000 7ff7ae689980 20 prepare_atomic_modification: state is not atomic. state=0x55c4e55e54d8 2024-05-09T20:11:59.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.354+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:59.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.354+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:59.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:59.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:59.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-05-09T20:11:59.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 olh_log_entry: epoch=2 op=3 key=e[UwnrChZpkzG6Pjh-scLXPWw57QKfmnC] 2024-05-09T20:11:59.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e528af60 obj=check-bucket:_:UwnrChZpkzG6Pjh-scLXPWw57QKfmnC_e state=0x55c4e55dd968 s->prefetch_data=0 2024-05-09T20:11:59.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-05-09T20:11:59.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 olh_log_entry: epoch=2 op=3 key=f[Bjtm1piNGHHZCNh9PRt5Z1ljt9.CJtf] 2024-05-09T20:11:59.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e5544770 obj=check-bucket:_:Bjtm1piNGHHZCNh9PRt5Z1ljt9.CJtf_f state=0x55c4e55df938 s->prefetch_data=0 2024-05-09T20:11:59.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 olh_log_entry: epoch=3 op=3 key=c[pakcFaOMMq76dUw353v2vfFXqPob7aD] 2024-05-09T20:11:59.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e55433c0 obj=check-bucket:_:pakcFaOMMq76dUw353v2vfFXqPob7aD_c state=0x55c4e55f5018 s->prefetch_data=0 2024-05-09T20:11:59.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 10 manifest: total_size = 11 2024-05-09T20:11:59.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4879206031877791745 2024-05-09T20:11:59.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e528af60 obj=check-bucket:_:UwnrChZpkzG6Pjh-scLXPWw57QKfmnC_e state=0x55c4e55dd968 s->prefetch_data=0 2024-05-09T20:11:59.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 prepare_atomic_modification: state is not atomic. state=0x55c4e55dd968 2024-05-09T20:11:59.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 10 manifest: total_size = 22 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16670127791563888858 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e5544770 obj=check-bucket:_:Bjtm1piNGHHZCNh9PRt5Z1ljt9.CJtf_f state=0x55c4e55df938 s->prefetch_data=0 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 prepare_atomic_modification: state is not atomic. state=0x55c4e55df938 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 10 manifest: total_size = 9 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1683333949890470770 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e55433c0 obj=check-bucket:_:pakcFaOMMq76dUw353v2vfFXqPob7aD_c state=0x55c4e55f5018 s->prefetch_data=0 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 prepare_atomic_modification: state is not atomic. state=0x55c4e55f5018 2024-05-09T20:11:59.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.358+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:59.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.366+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:59.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.366+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:59.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.366+0000 7ff6837fe640 20 handle_completion(): completion ok for obj=d[AXUzmGZFMkuZrYaKvH7t2aVaYHOlzQm] 2024-05-09T20:11:59.367 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.370+0000 7ff6837fe640 20 handle_completion(): completion ok for obj=e[UwnrChZpkzG6Pjh-scLXPWw57QKfmnC] 2024-05-09T20:11:59.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.374+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e528af60 obj=check-bucket:e state=0x55c4e5539888 s->prefetch_data=0 2024-05-09T20:11:59.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.374+0000 7ff7ae689980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=a2q7k212tjvhfcka067g5xxuuh2jc37o 2024-05-09T20:11:59.370 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.374+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:11:59.370 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.374+0000 7ff7ae689980 1 NOTICE: finished shard 5 (1 entries removed) 2024-05-09T20:11:59.371 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.374+0000 7ff6837fe640 20 handle_completion(): completion ok for obj=f[Bjtm1piNGHHZCNh9PRt5Z1ljt9.CJtf] 2024-05-09T20:11:59.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.378+0000 7ff7ae689980 20 get_obj_state: octx=0x55c4e5544770 obj=check-bucket:f state=0x55c4e553af48 s->prefetch_data=0 2024-05-09T20:11:59.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.378+0000 7ff7ae689980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=3ptohk6b9t6892whi9q475rspodp3yln 2024-05-09T20:11:59.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.378+0000 7ff6837fe640 20 handle_completion(): completion ok for obj=c[pakcFaOMMq76dUw353v2vfFXqPob7aD] 2024-05-09T20:11:59.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.378+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:59.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.378+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:59.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.386+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:11:59.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.386+0000 7ff7ae689980 1 NOTICE: finished shard 8 (1 entries removed) 2024-05-09T20:11:59.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.386+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:59.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.386+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:59.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.390+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:11:59.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.390+0000 7ff7ae689980 1 NOTICE: finished shard 10 (1 entries removed) 2024-05-09T20:11:59.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.390+0000 7ff7ae689980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:11:59.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.390+0000 7ff7ae689980 1 NOTICE: finished shard 1 (1 entries removed) 2024-05-09T20:11:59.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.390+0000 7ff7ae689980 1 NOTICE: finished all shards (4 entries removed) 2024-05-09T20:11:59.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.394+0000 7ff662fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:11:59.393 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.398+0000 7ff7ae689980 20 remove_watcher() i=0 2024-05-09T20:11:59.393 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.398+0000 7ff7ae689980 2 removed watcher, disabling cache 2024-05-09T20:11:59.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.398+0000 7ff7ae689980 20 remove_watcher() i=1 2024-05-09T20:11:59.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.402+0000 7ff7ae689980 20 remove_watcher() i=2 2024-05-09T20:11:59.400 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.406+0000 7ff7ae689980 20 remove_watcher() i=3 2024-05-09T20:11:59.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.406+0000 7ff7ae689980 20 remove_watcher() i=4 2024-05-09T20:11:59.405 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.410+0000 7ff7ae689980 20 remove_watcher() i=5 2024-05-09T20:11:59.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.410+0000 7ff7ae689980 20 remove_watcher() i=6 2024-05-09T20:11:59.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:11:59.414+0000 7ff7ae689980 20 remove_watcher() i=7 2024-05-09T20:11:59.656 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check unlinked does not affect normal entries 2024-05-09T20:11:59.656 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:12:00.074 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check unlinked does not find new unlistable entries after fix 2024-05-09T20:12:00.074 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:12:00.075 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --min-age-hours 0 --dump-keys 2024-05-09T20:12:00.113 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:00.113 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:00.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.162+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 realm 2024-05-09T20:12:00.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:00.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:00.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:00.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.166+0000 7fe3802a9980 20 searching for the correct realm 2024-05-09T20:12:00.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.170+0000 7fe37e5be640 0 -- 172.21.15.63:0/2621340222 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fe374025070 msgr2=0x7fe37403f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:00.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.178+0000 7fe37d5bc640 0 -- 172.21.15.63:0/2621340222 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fe37003f040 msgr2=0x7fe370025450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:00.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.186+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 zone default found 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 4 Realm: () 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 zonegroup default 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 started sync module instance, tier type = 2024-05-09T20:12:00.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.190+0000 7fe3802a9980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:00.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 20 add_watcher() i=0 2024-05-09T20:12:00.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 20 add_watcher() i=1 2024-05-09T20:12:00.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 20 add_watcher() i=2 2024-05-09T20:12:00.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 20 add_watcher() i=3 2024-05-09T20:12:00.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 20 add_watcher() i=4 2024-05-09T20:12:00.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 20 add_watcher() i=5 2024-05-09T20:12:00.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 20 add_watcher() i=6 2024-05-09T20:12:00.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 20 add_watcher() i=7 2024-05-09T20:12:00.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.206+0000 7fe3802a9980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:00.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe22f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:00.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:00.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:00.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 5 note: GC not initialized 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe22e7fc640 20 reqs_thread_entry: start 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 Filter name: none 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe19ffff640 20 reqs_thread_entry: start 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:12:00.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:00.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:12:00.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:12:00.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:12:00.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:12:00.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:12:00.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.210+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:00.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-09T20:12:00.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:00.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 1 NOTICE: finished shard 1 (0 entries found) 2024-05-09T20:12:00.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:00.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-09T20:12:00.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:00.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-09T20:12:00.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:00.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-09T20:12:00.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.214+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:00.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 1 NOTICE: finished shard 5 (0 entries found) 2024-05-09T20:12:00.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:00.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-09T20:12:00.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:00.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-09T20:12:00.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:00.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 1 NOTICE: finished shard 8 (0 entries found) 2024-05-09T20:12:00.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:00.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-09T20:12:00.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:00.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 1 NOTICE: finished shard 10 (0 entries found) 2024-05-09T20:12:00.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe3802a9980 1 NOTICE: finished all shards (0 entries found) 2024-05-09T20:12:00.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.218+0000 7fe22f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:00.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.226+0000 7fe3802a9980 20 remove_watcher() i=0 2024-05-09T20:12:00.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.226+0000 7fe3802a9980 2 removed watcher, disabling cache 2024-05-09T20:12:00.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.226+0000 7fe3802a9980 20 remove_watcher() i=1 2024-05-09T20:12:00.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.230+0000 7fe3802a9980 20 remove_watcher() i=2 2024-05-09T20:12:00.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.230+0000 7fe3802a9980 20 remove_watcher() i=3 2024-05-09T20:12:00.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.234+0000 7fe3802a9980 20 remove_watcher() i=4 2024-05-09T20:12:00.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.234+0000 7fe3802a9980 20 remove_watcher() i=5 2024-05-09T20:12:00.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.234+0000 7fe3802a9980 20 remove_watcher() i=6 2024-05-09T20:12:00.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.238+0000 7fe3802a9980 20 remove_watcher() i=7 2024-05-09T20:12:00.639 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-05-09T20:12:00.672 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:00.672 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:00.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 20 realm 2024-05-09T20:12:00.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:00.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.722+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.726+0000 7f31eb533980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:00.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.726+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.726+0000 7f31eb533980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:00.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.726+0000 7f31eb533980 20 searching for the correct realm 2024-05-09T20:12:00.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.730+0000 7f31e9848640 0 -- 172.21.15.63:0/3546748169 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f31d40ce920 msgr2=0x7f31d40f8400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:00.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.734+0000 7f31e8846640 0 -- 172.21.15.63:0/3546748169 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f31dc00d030 msgr2=0x7f31dc00b7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:00.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:00.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 zone default found 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 4 Realm: () 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 zonegroup default 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 started sync module instance, tier type = 2024-05-09T20:12:00.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.746+0000 7f31eb533980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:00.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 20 add_watcher() i=0 2024-05-09T20:12:00.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 20 add_watcher() i=1 2024-05-09T20:12:00.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 20 add_watcher() i=2 2024-05-09T20:12:00.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 20 add_watcher() i=3 2024-05-09T20:12:00.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 20 add_watcher() i=4 2024-05-09T20:12:00.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 20 add_watcher() i=5 2024-05-09T20:12:00.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 20 add_watcher() i=6 2024-05-09T20:12:00.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 20 add_watcher() i=7 2024-05-09T20:12:00.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.762+0000 7f31eb533980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:00.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f309ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:00.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f31eb533980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:00.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f31eb533980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:00.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f31eb533980 5 note: GC not initialized 2024-05-09T20:12:00.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f309efe5640 20 reqs_thread_entry: start 2024-05-09T20:12:00.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f31eb533980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:00.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f31eb533980 20 Filter name: none 2024-05-09T20:12:00.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f309cfe1640 20 reqs_thread_entry: start 2024-05-09T20:12:00.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.766+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:00.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:00.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:00.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:12:00.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 INFO: main: starting shard=0 2024-05-09T20:12:00.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:00.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 INFO: main: starting shard=1 2024-05-09T20:12:00.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:00.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 INFO: main: starting shard=2 2024-05-09T20:12:00.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:00.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 INFO: main: starting shard=3 2024-05-09T20:12:00.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.770+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:00.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: starting shard=4 2024-05-09T20:12:00.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:00.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: starting shard=5 2024-05-09T20:12:00.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:00.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-05-09T20:12:00.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: starting shard=6 2024-05-09T20:12:00.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:00.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: starting shard=7 2024-05-09T20:12:00.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:00.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 INFO: main: starting shard=8 2024-05-09T20:12:00.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.774+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:00.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f31eb533980 20 INFO: main: starting shard=9 2024-05-09T20:12:00.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:00.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f31eb533980 20 INFO: main: starting shard=10 2024-05-09T20:12:00.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f31eb533980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:00.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f31eb533980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f31eb533980 20 INFO: main: done 2024-05-09T20:12:00.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.778+0000 7f309ffe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:00.779 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.782+0000 7f31eb533980 20 remove_watcher() i=0 2024-05-09T20:12:00.779 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.782+0000 7f31eb533980 2 removed watcher, disabling cache 2024-05-09T20:12:00.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.786+0000 7f31eb533980 20 remove_watcher() i=1 2024-05-09T20:12:00.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.786+0000 7f31eb533980 20 remove_watcher() i=2 2024-05-09T20:12:00.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.790+0000 7f31eb533980 20 remove_watcher() i=3 2024-05-09T20:12:00.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.794+0000 7f31eb533980 20 remove_watcher() i=4 2024-05-09T20:12:00.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.794+0000 7f31eb533980 20 remove_watcher() i=5 2024-05-09T20:12:00.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.794+0000 7f31eb533980 20 remove_watcher() i=6 2024-05-09T20:12:00.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.794+0000 7f31eb533980 20 remove_watcher() i=7 2024-05-09T20:12:00.805 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-05-09T20:12:00.833 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:00.833 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:00.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.878+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 realm 2024-05-09T20:12:00.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:00.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:00.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.882+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.886+0000 7ff752b70980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:00.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.886+0000 7ff752b70980 20 searching for the correct realm 2024-05-09T20:12:00.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.886+0000 7ff750e85640 0 -- 172.21.15.63:0/2886650960 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7ff744048040 msgr2=0x7ff744048420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:00.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.894+0000 7ff74fe83640 0 -- 172.21.15.63:0/2886650960 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7ff7400ca070 msgr2=0x7ff7400f1610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:00.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:00.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:00.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:00.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:00.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:00.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:00.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 zone default found 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 4 Realm: () 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 zonegroup default 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:00.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 started sync module instance, tier type = 2024-05-09T20:12:00.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.906+0000 7ff752b70980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:00.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 20 add_watcher() i=0 2024-05-09T20:12:00.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 20 add_watcher() i=1 2024-05-09T20:12:00.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 20 add_watcher() i=2 2024-05-09T20:12:00.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 20 add_watcher() i=3 2024-05-09T20:12:00.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 20 add_watcher() i=4 2024-05-09T20:12:00.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 20 add_watcher() i=5 2024-05-09T20:12:00.922 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 20 add_watcher() i=6 2024-05-09T20:12:00.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 20 add_watcher() i=7 2024-05-09T20:12:00.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.926+0000 7ff752b70980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:00.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff606fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:00.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:00.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:00.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 5 note: GC not initialized 2024-05-09T20:12:00.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff605fdb640 20 reqs_thread_entry: start 2024-05-09T20:12:00.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:00.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 Filter name: none 2024-05-09T20:12:00.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff5777fe640 20 reqs_thread_entry: start 2024-05-09T20:12:00.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:00.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 rados->read ofs=0 len=0 2024-05-09T20:12:00.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:00.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:00.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:12:00.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 INFO: main: starting shard=0 2024-05-09T20:12:00.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.930+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:00.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: starting shard=1 2024-05-09T20:12:00.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:00.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: starting shard=2 2024-05-09T20:12:00.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:00.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: starting shard=3 2024-05-09T20:12:00.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:00.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: starting shard=4 2024-05-09T20:12:00.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:00.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 INFO: main: starting shard=5 2024-05-09T20:12:00.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.934+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:00.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: starting shard=6 2024-05-09T20:12:00.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:00.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: starting shard=7 2024-05-09T20:12:00.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:00.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: starting shard=8 2024-05-09T20:12:00.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:00.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=2, is_truncated=0, marker= 2024-05-09T20:12:00.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: starting shard=9 2024-05-09T20:12:00.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:00.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: starting shard=10 2024-05-09T20:12:00.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:00.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:00.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.938+0000 7ff752b70980 20 INFO: main: done 2024-05-09T20:12:00.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.942+0000 7ff606fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:00.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.946+0000 7ff752b70980 20 remove_watcher() i=0 2024-05-09T20:12:00.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.946+0000 7ff752b70980 2 removed watcher, disabling cache 2024-05-09T20:12:00.943 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.946+0000 7ff752b70980 20 remove_watcher() i=1 2024-05-09T20:12:00.946 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.950+0000 7ff752b70980 20 remove_watcher() i=2 2024-05-09T20:12:00.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.954+0000 7ff752b70980 20 remove_watcher() i=3 2024-05-09T20:12:00.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.954+0000 7ff752b70980 20 remove_watcher() i=4 2024-05-09T20:12:00.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.954+0000 7ff752b70980 20 remove_watcher() i=5 2024-05-09T20:12:00.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.958+0000 7ff752b70980 20 remove_watcher() i=6 2024-05-09T20:12:00.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:00.962+0000 7ff752b70980 20 remove_watcher() i=7 2024-05-09T20:12:00.970 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check olh finds leftover OLH entries 2024-05-09T20:12:00.970 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:12:00.970 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-05-09T20:12:00.999 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:00.999 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:01.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 20 realm 2024-05-09T20:12:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.046+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.050+0000 7fcee7b28980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:01.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.050+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.050+0000 7fcee7b28980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:01.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.050+0000 7fcee7b28980 20 searching for the correct realm 2024-05-09T20:12:01.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.054+0000 7fcee563c640 0 -- 172.21.15.63:0/1490766251 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fcee005c6a0 msgr2=0x7fcee005b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:01.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.270+0000 7fcee563c640 0 -- 172.21.15.63:0/1490766251 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fcee005c6a0 msgr2=0x7fcee005b300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:01.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:01.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:01.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:01.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:01.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:01.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:01.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:01.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.282+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 zone default found 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 4 Realm: () 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 zonegroup default 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 started sync module instance, tier type = 2024-05-09T20:12:01.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.286+0000 7fcee7b28980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:01.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 20 add_watcher() i=0 2024-05-09T20:12:01.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 20 add_watcher() i=1 2024-05-09T20:12:01.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 20 add_watcher() i=2 2024-05-09T20:12:01.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 20 add_watcher() i=3 2024-05-09T20:12:01.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 20 add_watcher() i=4 2024-05-09T20:12:01.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 20 add_watcher() i=5 2024-05-09T20:12:01.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 20 add_watcher() i=6 2024-05-09T20:12:01.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 20 add_watcher() i=7 2024-05-09T20:12:01.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.314+0000 7fcee7b28980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:01.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.318+0000 7fcd9b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:01.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.318+0000 7fcee7b28980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:01.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.318+0000 7fcee7b28980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:01.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.318+0000 7fcee7b28980 5 note: GC not initialized 2024-05-09T20:12:01.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcd9a7e4640 20 reqs_thread_entry: start 2024-05-09T20:12:01.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:01.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 Filter name: none 2024-05-09T20:12:01.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcd0bfff640 20 reqs_thread_entry: start 2024-05-09T20:12:01.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:12:01.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:12:01.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:12:01.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:01.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-09T20:12:01.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:01.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 1 NOTICE: finished shard 1 (0 entries found) 2024-05-09T20:12:01.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.322+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:01.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.326+0000 7fcee7b28980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-09T20:12:01.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.326+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:01.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.326+0000 7fcee7b28980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-09T20:12:01.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.326+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:01.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.326+0000 7fcee7b28980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-09T20:12:01.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.326+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:01.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.326+0000 7fcee7b28980 1 NOTICE: finished shard 5 (1 entries found) 2024-05-09T20:12:01.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.326+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:01.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-09T20:12:01.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:01.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-09T20:12:01.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:01.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 1 NOTICE: finished shard 8 (1 entries found) 2024-05-09T20:12:01.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:01.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-09T20:12:01.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:01.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 1 NOTICE: finished shard 10 (0 entries found) 2024-05-09T20:12:01.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.330+0000 7fcee7b28980 1 NOTICE: finished all shards (2 entries found) 2024-05-09T20:12:01.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.334+0000 7fcd9b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:01.333 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.338+0000 7fcee7b28980 20 remove_watcher() i=0 2024-05-09T20:12:01.333 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.338+0000 7fcee7b28980 2 removed watcher, disabling cache 2024-05-09T20:12:01.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.342+0000 7fcee7b28980 20 remove_watcher() i=1 2024-05-09T20:12:01.339 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.342+0000 7fcee7b28980 20 remove_watcher() i=2 2024-05-09T20:12:01.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.346+0000 7fcee7b28980 20 remove_watcher() i=3 2024-05-09T20:12:01.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.346+0000 7fcee7b28980 20 remove_watcher() i=4 2024-05-09T20:12:01.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.350+0000 7fcee7b28980 20 remove_watcher() i=5 2024-05-09T20:12:01.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.350+0000 7fcee7b28980 20 remove_watcher() i=6 2024-05-09T20:12:01.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.354+0000 7fcee7b28980 20 remove_watcher() i=7 2024-05-09T20:12:01.365 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check olh fixes leftover OLH entries 2024-05-09T20:12:01.365 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:12:01.366 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check olh --bucket check-bucket --fix --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-05-09T20:12:01.398 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:01.398 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:01.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 20 realm 2024-05-09T20:12:01.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:01.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.450+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.454+0000 7f8c3020c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:01.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.454+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.454+0000 7f8c3020c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:01.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.454+0000 7f8c3020c980 20 searching for the correct realm 2024-05-09T20:12:01.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.458+0000 7f8c2dd20640 0 -- 172.21.15.63:0/1121999988 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f8c28067b40 msgr2=0x7f8c28064930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:01.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.674+0000 7f8c2dd20640 0 -- 172.21.15.63:0/1121999988 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f8c28067b40 msgr2=0x7f8c28064930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:01.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.686+0000 7f8c3020c980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:01.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.686+0000 7f8c3020c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:01.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.686+0000 7f8c3020c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:01.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.686+0000 7f8c3020c980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:01.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.686+0000 7f8c3020c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:01.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.686+0000 7f8c3020c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:01.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.686+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:01.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:01.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 zone default found 2024-05-09T20:12:01.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 4 Realm: () 2024-05-09T20:12:01.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:01.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:01.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:01.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 zonegroup default 2024-05-09T20:12:01.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.690+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.694+0000 7f8c3020c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.694+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.694+0000 7f8c3020c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.694+0000 7f8c3020c980 20 started sync module instance, tier type = 2024-05-09T20:12:01.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.694+0000 7f8c3020c980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:01.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 20 add_watcher() i=0 2024-05-09T20:12:01.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 20 add_watcher() i=1 2024-05-09T20:12:01.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 20 add_watcher() i=2 2024-05-09T20:12:01.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 20 add_watcher() i=3 2024-05-09T20:12:01.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 20 add_watcher() i=4 2024-05-09T20:12:01.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 20 add_watcher() i=5 2024-05-09T20:12:01.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 20 add_watcher() i=6 2024-05-09T20:12:01.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 20 add_watcher() i=7 2024-05-09T20:12:01.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.722+0000 7f8c3020c980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:01.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.726+0000 7f8adf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:01.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:01.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:01.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 5 note: GC not initialized 2024-05-09T20:12:01.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8ade7fc640 20 reqs_thread_entry: start 2024-05-09T20:12:01.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:01.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 20 Filter name: none 2024-05-09T20:12:01.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8a4ffff640 20 reqs_thread_entry: start 2024-05-09T20:12:01.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:12:01.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:01.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:12:01.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:12:01.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:12:01.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.730+0000 7f8c3020c980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:01.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:12:01.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:12:01.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:12:01.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:12:01.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:12:01.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:01.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 1 NOTICE: finished shard 0 (0 entries removed) 2024-05-09T20:12:01.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:01.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 1 NOTICE: finished shard 1 (0 entries removed) 2024-05-09T20:12:01.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:01.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 1 NOTICE: finished shard 2 (0 entries removed) 2024-05-09T20:12:01.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.734+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:01.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.738+0000 7f8c3020c980 1 NOTICE: finished shard 3 (0 entries removed) 2024-05-09T20:12:01.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.738+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:01.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.738+0000 7f8c3020c980 1 NOTICE: finished shard 4 (0 entries removed) 2024-05-09T20:12:01.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.738+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:01.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.738+0000 7f8c3020c980 20 get_obj_state: octx=0x55809a5c0f50 obj=check-bucket:d state=0x55809a871248 s->prefetch_data=0 2024-05-09T20:12:01.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.738+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:01.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 1 NOTICE: finished shard 6 (0 entries removed) 2024-05-09T20:12:01.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:01.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 1 NOTICE: finished shard 7 (0 entries removed) 2024-05-09T20:12:01.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:01.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 get_obj_state: octx=0x55809a570290 obj=check-bucket:c state=0x55809a8726c8 s->prefetch_data=0 2024-05-09T20:12:01.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:01.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 1 NOTICE: finished shard 9 (0 entries removed) 2024-05-09T20:12:01.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:01.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 1 NOTICE: finished shard 10 (0 entries removed) 2024-05-09T20:12:01.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 get_obj_state: setting s->obj_tag to c47i492hokhl3ozh78b1w4pt06k2wavw 2024-05-09T20:12:01.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 get_obj_state_impl: setting s->olh_tag to x6r4t0f071lcr3ofcquyb57cxnz3k4ss 2024-05-09T20:12:01.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:01.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 get_obj_state: setting s->obj_tag to xpvisso0krd5y2i368evh4ewrljknr3r 2024-05-09T20:12:01.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 get_obj_state_impl: setting s->olh_tag to tp6l7gk1e302gcdwlxwis3qc3u4ftbtk 2024-05-09T20:12:01.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:01.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 olh_log_entry: epoch=4 op=2 key=c[] 2024-05-09T20:12:01.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 olh_log_entry: epoch=4 op=3 key=c[l-ivPyOLjLC4fcPr03oS7.cEpMBlWvY] 2024-05-09T20:12:01.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 get_obj_state: octx=0x55809a8c41b0 obj=check-bucket:_:l-ivPyOLjLC4fcPr03oS7.cEpMBlWvY_c state=0x55809a8ca988 s->prefetch_data=0 2024-05-09T20:12:01.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 olh_log_entry: epoch=4 op=2 key=d[] 2024-05-09T20:12:01.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.742+0000 7f8c3020c980 20 olh_log_entry: epoch=4 op=3 key=d[-onlglx09ubkjL.9DsxCEf3YNe.k.76] 2024-05-09T20:12:01.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.746+0000 7f8c3020c980 20 get_obj_state: octx=0x55809a8a40b0 obj=check-bucket:_:-onlglx09ubkjL.9DsxCEf3YNe.k.76_d state=0x55809a8cf9d8 s->prefetch_data=0 2024-05-09T20:12:01.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.746+0000 7f8c3020c980 20 get_obj_state: octx=0x55809a8c41b0 obj=check-bucket:c state=0x55809a874898 s->prefetch_data=0 2024-05-09T20:12:01.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.746+0000 7f8c3020c980 20 get_obj_state: setting s->obj_tag to xpvisso0krd5y2i368evh4ewrljknr3r 2024-05-09T20:12:01.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.746+0000 7f8c3020c980 20 get_obj_state_impl: setting s->olh_tag to tp6l7gk1e302gcdwlxwis3qc3u4ftbtk 2024-05-09T20:12:01.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.746+0000 7f8c3020c980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=tp6l7gk1e302gcdwlxwis3qc3u4ftbtk 2024-05-09T20:12:01.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.746+0000 7f8c3020c980 20 get_obj_state: octx=0x55809a8a40b0 obj=check-bucket:d state=0x55809a875f08 s->prefetch_data=0 2024-05-09T20:12:01.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.750+0000 7f8c3020c980 20 get_obj_state: setting s->obj_tag to c47i492hokhl3ozh78b1w4pt06k2wavw 2024-05-09T20:12:01.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.750+0000 7f8c3020c980 20 get_obj_state_impl: setting s->olh_tag to x6r4t0f071lcr3ofcquyb57cxnz3k4ss 2024-05-09T20:12:01.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.750+0000 7f8c3020c980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=x6r4t0f071lcr3ofcquyb57cxnz3k4ss 2024-05-09T20:12:01.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.754+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:01.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.758+0000 7f8c3020c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:01.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.758+0000 7f8c3020c980 1 NOTICE: finished shard 8 (1 entries removed) 2024-05-09T20:12:01.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.762+0000 7f8c3020c980 1 NOTICE: finished shard 5 (1 entries removed) 2024-05-09T20:12:01.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.762+0000 7f8c3020c980 1 NOTICE: finished all shards (2 entries removed) 2024-05-09T20:12:01.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.762+0000 7f8adf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:01.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.770+0000 7f8c3020c980 20 remove_watcher() i=0 2024-05-09T20:12:01.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.770+0000 7f8c3020c980 2 removed watcher, disabling cache 2024-05-09T20:12:01.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.770+0000 7f8c3020c980 20 remove_watcher() i=1 2024-05-09T20:12:01.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.774+0000 7f8c3020c980 20 remove_watcher() i=2 2024-05-09T20:12:01.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.774+0000 7f8c3020c980 20 remove_watcher() i=3 2024-05-09T20:12:01.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.778+0000 7f8c3020c980 20 remove_watcher() i=4 2024-05-09T20:12:01.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.778+0000 7f8c3020c980 20 remove_watcher() i=5 2024-05-09T20:12:01.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.782+0000 7f8c3020c980 20 remove_watcher() i=6 2024-05-09T20:12:01.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.786+0000 7f8c3020c980 20 remove_watcher() i=7 2024-05-09T20:12:01.800 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-05-09T20:12:01.829 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:01.829 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:01.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 20 realm 2024-05-09T20:12:01.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:01.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:01.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.878+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.882+0000 7f3d94b8e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:01.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.882+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:01.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.882+0000 7f3d94b8e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:01.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.882+0000 7f3d94b8e980 20 searching for the correct realm 2024-05-09T20:12:01.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:01.886+0000 7f3d92ea3640 0 -- 172.21.15.63:0/2838210718 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f3d88118020 msgr2=0x7f3d880ef960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:12:02.095 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.098+0000 7f3d91ea1640 0 -- 172.21.15.63:0/2838210718 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f3d8c022040 msgr2=0x7f3d8c0163f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:02.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.114+0000 7f3d94b8e980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:02.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.114+0000 7f3d94b8e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.114+0000 7f3d94b8e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.114+0000 7f3d94b8e980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.114+0000 7f3d94b8e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.114+0000 7f3d94b8e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.114+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:02.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:02.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 zone default found 2024-05-09T20:12:02.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 4 Realm: () 2024-05-09T20:12:02.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:02.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:02.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:02.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 zonegroup default 2024-05-09T20:12:02.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.118+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.122+0000 7f3d94b8e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.122+0000 7f3d94b8e980 20 started sync module instance, tier type = 2024-05-09T20:12:02.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.122+0000 7f3d94b8e980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.146+0000 7f3d94b8e980 20 add_watcher() i=0 2024-05-09T20:12:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.146+0000 7f3d94b8e980 20 add_watcher() i=1 2024-05-09T20:12:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.146+0000 7f3d94b8e980 20 add_watcher() i=2 2024-05-09T20:12:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.146+0000 7f3d94b8e980 20 add_watcher() i=3 2024-05-09T20:12:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.146+0000 7f3d94b8e980 20 add_watcher() i=4 2024-05-09T20:12:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.146+0000 7f3d94b8e980 20 add_watcher() i=5 2024-05-09T20:12:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.146+0000 7f3d94b8e980 20 add_watcher() i=6 2024-05-09T20:12:02.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.150+0000 7f3d94b8e980 20 add_watcher() i=7 2024-05-09T20:12:02.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.150+0000 7f3d94b8e980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:02.149 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3c48fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:02.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3d94b8e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:02.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3d94b8e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:02.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3d94b8e980 5 note: GC not initialized 2024-05-09T20:12:02.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3bc37fe640 20 reqs_thread_entry: start 2024-05-09T20:12:02.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3d94b8e980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:02.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3d94b8e980 20 Filter name: none 2024-05-09T20:12:02.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3bc17fa640 20 reqs_thread_entry: start 2024-05-09T20:12:02.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3d94b8e980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:02.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.154+0000 7f3d94b8e980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:02.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:02.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:12:02.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 INFO: main: starting shard=0 2024-05-09T20:12:02.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:02.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 INFO: main: starting shard=1 2024-05-09T20:12:02.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:02.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 INFO: main: starting shard=2 2024-05-09T20:12:02.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:02.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 INFO: main: starting shard=3 2024-05-09T20:12:02.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.158+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:02.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 INFO: main: starting shard=4 2024-05-09T20:12:02.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:02.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 INFO: main: starting shard=5 2024-05-09T20:12:02.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:02.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 INFO: main: starting shard=6 2024-05-09T20:12:02.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:02.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 INFO: main: starting shard=7 2024-05-09T20:12:02.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.162+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:02.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 INFO: main: starting shard=8 2024-05-09T20:12:02.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:02.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 INFO: main: starting shard=9 2024-05-09T20:12:02.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:02.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 INFO: main: starting shard=10 2024-05-09T20:12:02.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:02.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3d94b8e980 20 INFO: main: done 2024-05-09T20:12:02.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.166+0000 7f3c48fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:02.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.174+0000 7f3d94b8e980 20 remove_watcher() i=0 2024-05-09T20:12:02.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.174+0000 7f3d94b8e980 2 removed watcher, disabling cache 2024-05-09T20:12:02.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.174+0000 7f3d94b8e980 20 remove_watcher() i=1 2024-05-09T20:12:02.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.178+0000 7f3d94b8e980 20 remove_watcher() i=2 2024-05-09T20:12:02.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.182+0000 7f3d94b8e980 20 remove_watcher() i=3 2024-05-09T20:12:02.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.182+0000 7f3d94b8e980 20 remove_watcher() i=4 2024-05-09T20:12:02.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.186+0000 7f3d94b8e980 20 remove_watcher() i=5 2024-05-09T20:12:02.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.186+0000 7f3d94b8e980 20 remove_watcher() i=6 2024-05-09T20:12:02.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.186+0000 7f3d94b8e980 20 remove_watcher() i=7 2024-05-09T20:12:02.201 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-05-09T20:12:02.231 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:02.231 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:02.275 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.278+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 realm 2024-05-09T20:12:02.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:02.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:02.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.282+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.286+0000 7f69f53ec980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:02.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.286+0000 7f69f53ec980 20 searching for the correct realm 2024-05-09T20:12:02.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.290+0000 7f69f2f00640 0 -- 172.21.15.63:0/3339831903 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f69ec0dd040 msgr2=0x7f69ec0d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:12:02.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.506+0000 7f69f2f00640 0 -- 172.21.15.63:0/3339831903 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f69ec0dd040 msgr2=0x7f69ec0d0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:02.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:02.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:02.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:02.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:02.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:02.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:02.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:02.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.518+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:02.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 zone default found 2024-05-09T20:12:02.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 4 Realm: () 2024-05-09T20:12:02.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:02.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:02.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:02.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 zonegroup default 2024-05-09T20:12:02.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 started sync module instance, tier type = 2024-05-09T20:12:02.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.522+0000 7f69f53ec980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:02.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 20 add_watcher() i=0 2024-05-09T20:12:02.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 20 add_watcher() i=1 2024-05-09T20:12:02.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 20 add_watcher() i=2 2024-05-09T20:12:02.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 20 add_watcher() i=3 2024-05-09T20:12:02.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 20 add_watcher() i=4 2024-05-09T20:12:02.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 20 add_watcher() i=5 2024-05-09T20:12:02.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 20 add_watcher() i=6 2024-05-09T20:12:02.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 20 add_watcher() i=7 2024-05-09T20:12:02.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.550+0000 7f69f53ec980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:02.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.554+0000 7f68a97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:02.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:02.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:02.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 5 note: GC not initialized 2024-05-09T20:12:02.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f6823fff640 20 reqs_thread_entry: start 2024-05-09T20:12:02.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:02.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 Filter name: none 2024-05-09T20:12:02.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f6821ffb640 20 reqs_thread_entry: start 2024-05-09T20:12:02.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:02.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:02.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:02.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:12:02.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 INFO: main: starting shard=0 2024-05-09T20:12:02.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.558+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:02.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 INFO: main: starting shard=1 2024-05-09T20:12:02.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:02.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 INFO: main: starting shard=2 2024-05-09T20:12:02.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:02.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 INFO: main: starting shard=3 2024-05-09T20:12:02.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:02.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 INFO: main: starting shard=4 2024-05-09T20:12:02.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.562+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:02.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 INFO: main: starting shard=5 2024-05-09T20:12:02.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:02.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 INFO: main: starting shard=6 2024-05-09T20:12:02.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:02.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 INFO: main: starting shard=7 2024-05-09T20:12:02.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:02.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 INFO: main: starting shard=8 2024-05-09T20:12:02.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.566+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:02.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f69f53ec980 20 INFO: main: starting shard=9 2024-05-09T20:12:02.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:02.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f69f53ec980 20 INFO: main: starting shard=10 2024-05-09T20:12:02.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f69f53ec980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:02.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f69f53ec980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:02.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f69f53ec980 20 INFO: main: done 2024-05-09T20:12:02.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.570+0000 7f68a97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:02.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.578+0000 7f69f53ec980 20 remove_watcher() i=0 2024-05-09T20:12:02.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.578+0000 7f69f53ec980 2 removed watcher, disabling cache 2024-05-09T20:12:02.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.582+0000 7f69f53ec980 20 remove_watcher() i=1 2024-05-09T20:12:02.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.582+0000 7f69f53ec980 20 remove_watcher() i=2 2024-05-09T20:12:02.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.586+0000 7f69f53ec980 20 remove_watcher() i=3 2024-05-09T20:12:02.583 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.586+0000 7f69f53ec980 20 remove_watcher() i=4 2024-05-09T20:12:02.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.590+0000 7f69f53ec980 20 remove_watcher() i=5 2024-05-09T20:12:02.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.590+0000 7f69f53ec980 20 remove_watcher() i=6 2024-05-09T20:12:02.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.594+0000 7f69f53ec980 20 remove_watcher() i=7 2024-05-09T20:12:02.605 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check olh does not find new leftover OLH entries after fix 2024-05-09T20:12:02.605 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:12:02.606 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check olh --bucket check-bucket --dump-keys 2024-05-09T20:12:02.635 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:02.635 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:02.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.682+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 realm 2024-05-09T20:12:02.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:02.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:02.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.686+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.690+0000 7f0b29837980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:02.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.690+0000 7f0b29837980 20 searching for the correct realm 2024-05-09T20:12:02.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.690+0000 7f0b27b4c640 0 -- 172.21.15.63:0/784629817 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f0b1c01e810 msgr2=0x7f0b1c06a0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:02.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.698+0000 7f0b26b4a640 0 -- 172.21.15.63:0/784629817 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f0b180b8bc0 msgr2=0x7f0b180d3920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.706+0000 7f0b29837980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.706+0000 7f0b29837980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.706+0000 7f0b29837980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.706+0000 7f0b29837980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.706+0000 7f0b29837980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.706+0000 7f0b29837980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.706+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:02.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:02.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 zone default found 2024-05-09T20:12:02.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 4 Realm: () 2024-05-09T20:12:02.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:02.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:02.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:02.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 zonegroup default 2024-05-09T20:12:02.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:02.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 started sync module instance, tier type = 2024-05-09T20:12:02.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.710+0000 7f0b29837980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:02.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.726+0000 7f0b29837980 20 add_watcher() i=0 2024-05-09T20:12:02.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.726+0000 7f0b29837980 20 add_watcher() i=1 2024-05-09T20:12:02.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.726+0000 7f0b29837980 20 add_watcher() i=2 2024-05-09T20:12:02.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.726+0000 7f0b29837980 20 add_watcher() i=3 2024-05-09T20:12:02.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.726+0000 7f0b29837980 20 add_watcher() i=4 2024-05-09T20:12:02.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.726+0000 7f0b29837980 20 add_watcher() i=5 2024-05-09T20:12:02.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.726+0000 7f0b29837980 20 add_watcher() i=6 2024-05-09T20:12:02.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.730+0000 7f0b29837980 20 add_watcher() i=7 2024-05-09T20:12:02.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.730+0000 7f0b29837980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:02.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.730+0000 7f09ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:02.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.730+0000 7f0b29837980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:02.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.730+0000 7f0b29837980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:02.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 5 note: GC not initialized 2024-05-09T20:12:02.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f09dcfe1640 20 reqs_thread_entry: start 2024-05-09T20:12:02.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:02.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 Filter name: none 2024-05-09T20:12:02.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f09567fc640 20 reqs_thread_entry: start 2024-05-09T20:12:02.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:12:02.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 rados->read ofs=0 len=0 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:12:02.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:12:02.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:02.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 1 NOTICE: finished shard 0 (0 entries found) 2024-05-09T20:12:02.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:02.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 1 NOTICE: finished shard 1 (0 entries found) 2024-05-09T20:12:02.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.734+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:02.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.738+0000 7f0b29837980 1 NOTICE: finished shard 2 (0 entries found) 2024-05-09T20:12:02.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.738+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:02.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.738+0000 7f0b29837980 1 NOTICE: finished shard 3 (0 entries found) 2024-05-09T20:12:02.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.738+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:02.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.738+0000 7f0b29837980 1 NOTICE: finished shard 4 (0 entries found) 2024-05-09T20:12:02.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.738+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:02.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.738+0000 7f0b29837980 1 NOTICE: finished shard 5 (0 entries found) 2024-05-09T20:12:02.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.738+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:02.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 1 NOTICE: finished shard 6 (0 entries found) 2024-05-09T20:12:02.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:02.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 1 NOTICE: finished shard 7 (0 entries found) 2024-05-09T20:12:02.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:02.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 1 NOTICE: finished shard 8 (0 entries found) 2024-05-09T20:12:02.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:02.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 1 NOTICE: finished shard 9 (0 entries found) 2024-05-09T20:12:02.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:02.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 1 NOTICE: finished shard 10 (0 entries found) 2024-05-09T20:12:02.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f0b29837980 1 NOTICE: finished all shards (0 entries found) 2024-05-09T20:12:02.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.742+0000 7f09ddfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:02.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.750+0000 7f0b29837980 20 remove_watcher() i=0 2024-05-09T20:12:02.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.750+0000 7f0b29837980 2 removed watcher, disabling cache 2024-05-09T20:12:02.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.750+0000 7f0b29837980 20 remove_watcher() i=1 2024-05-09T20:12:02.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.754+0000 7f0b29837980 20 remove_watcher() i=2 2024-05-09T20:12:02.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.754+0000 7f0b29837980 20 remove_watcher() i=3 2024-05-09T20:12:02.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.758+0000 7f0b29837980 20 remove_watcher() i=4 2024-05-09T20:12:02.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.758+0000 7f0b29837980 20 remove_watcher() i=5 2024-05-09T20:12:02.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.762+0000 7f0b29837980 20 remove_watcher() i=6 2024-05-09T20:12:02.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:02.762+0000 7f0b29837980 20 remove_watcher() i=7 2024-05-09T20:12:02.771 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: verify that bucket check fixes do not affect null version objects 2024-05-09T20:12:02.771 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:12:02.940 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: bucket check stats are correct in the presence of unlinked entries 2024-05-09T20:12:02.940 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:12:04.917 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-05-09T20:12:05.355 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-05-09T20:12:06.933 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:12:06.933 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object c 2024-05-09T20:12:06.969 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:06.969 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:07.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.018+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 realm 2024-05-09T20:12:07.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:07.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:07.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:07.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.022+0000 7f18f2fe2980 20 searching for the correct realm 2024-05-09T20:12:07.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.026+0000 7f18f12f7640 0 -- 172.21.15.63:0/2268554934 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f18e403fb10 msgr2=0x7f18e404f930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:07.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.034+0000 7f18f02f5640 0 -- 172.21.15.63:0/2268554934 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f18e005eac0 msgr2=0x7f18e0075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:07.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:07.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:07.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:07.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:07.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:07.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:07.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.042+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:07.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:07.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 zone default found 2024-05-09T20:12:07.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 4 Realm: () 2024-05-09T20:12:07.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:07.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:07.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:07.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 zonegroup default 2024-05-09T20:12:07.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 started sync module instance, tier type = 2024-05-09T20:12:07.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.046+0000 7f18f2fe2980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:07.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.070+0000 7f18f2fe2980 20 add_watcher() i=0 2024-05-09T20:12:07.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.070+0000 7f18f2fe2980 20 add_watcher() i=1 2024-05-09T20:12:07.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.070+0000 7f18f2fe2980 20 add_watcher() i=2 2024-05-09T20:12:07.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.070+0000 7f18f2fe2980 20 add_watcher() i=3 2024-05-09T20:12:07.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.070+0000 7f18f2fe2980 20 add_watcher() i=4 2024-05-09T20:12:07.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.070+0000 7f18f2fe2980 20 add_watcher() i=5 2024-05-09T20:12:07.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.070+0000 7f18f2fe2980 20 add_watcher() i=6 2024-05-09T20:12:07.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.074+0000 7f18f2fe2980 20 add_watcher() i=7 2024-05-09T20:12:07.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.074+0000 7f18f2fe2980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:07.073 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f17a6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:07.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f18f2fe2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:07.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f18f2fe2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:07.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f18f2fe2980 5 note: GC not initialized 2024-05-09T20:12:07.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f17a5fe3640 20 reqs_thread_entry: start 2024-05-09T20:12:07.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f18f2fe2980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:07.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f18f2fe2980 20 Filter name: none 2024-05-09T20:12:07.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f17177fe640 20 reqs_thread_entry: start 2024-05-09T20:12:07.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f18f2fe2980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:07.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.078+0000 7f18f2fe2980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:07.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:07.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:12:07.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 INFO: main: starting shard=0 2024-05-09T20:12:07.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:07.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 INFO: main: starting shard=1 2024-05-09T20:12:07.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:07.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 INFO: main: starting shard=2 2024-05-09T20:12:07.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:07.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 INFO: main: starting shard=3 2024-05-09T20:12:07.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.082+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:07.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 INFO: main: starting shard=4 2024-05-09T20:12:07.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:07.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 INFO: main: starting shard=5 2024-05-09T20:12:07.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:07.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 INFO: main: starting shard=6 2024-05-09T20:12:07.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:07.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 INFO: main: starting shard=7 2024-05-09T20:12:07.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.086+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:07.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 INFO: main: starting shard=8 2024-05-09T20:12:07.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:07.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-05-09T20:12:07.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 INFO: main: starting shard=9 2024-05-09T20:12:07.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:07.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 INFO: main: starting shard=10 2024-05-09T20:12:07.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:07.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f18f2fe2980 20 INFO: main: done 2024-05-09T20:12:07.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.090+0000 7f17a6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:07.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.098+0000 7f18f2fe2980 20 remove_watcher() i=0 2024-05-09T20:12:07.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.098+0000 7f18f2fe2980 2 removed watcher, disabling cache 2024-05-09T20:12:07.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.098+0000 7f18f2fe2980 20 remove_watcher() i=1 2024-05-09T20:12:07.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.102+0000 7f18f2fe2980 20 remove_watcher() i=2 2024-05-09T20:12:07.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.102+0000 7f18f2fe2980 20 remove_watcher() i=3 2024-05-09T20:12:07.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.106+0000 7f18f2fe2980 20 remove_watcher() i=4 2024-05-09T20:12:07.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.106+0000 7f18f2fe2980 20 remove_watcher() i=5 2024-05-09T20:12:07.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.106+0000 7f18f2fe2980 20 remove_watcher() i=6 2024-05-09T20:12:07.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.110+0000 7f18f2fe2980 20 remove_watcher() i=7 2024-05-09T20:12:07.269 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:12:07.269 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object d 2024-05-09T20:12:07.306 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:07.306 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:07.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.354+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 realm 2024-05-09T20:12:07.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:07.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:07.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:07.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.358+0000 7f08a874c980 20 searching for the correct realm 2024-05-09T20:12:07.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.362+0000 7f08a6260640 0 -- 172.21.15.63:0/4172062607 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f08a0041920 msgr2=0x7f08a0032400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:07.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.578+0000 7f08a6260640 0 -- 172.21.15.63:0/4172062607 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f08a0041920 msgr2=0x7f08a0032400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:07.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:07.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:07.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:07.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:07.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:07.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:07.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:07.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 zone default found 2024-05-09T20:12:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 4 Realm: () 2024-05-09T20:12:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 zonegroup default 2024-05-09T20:12:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.590+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.594+0000 7f08a874c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.594+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.594+0000 7f08a874c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.594+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.594+0000 7f08a874c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:07.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.594+0000 7f08a874c980 20 started sync module instance, tier type = 2024-05-09T20:12:07.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.594+0000 7f08a874c980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:07.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.618+0000 7f08a874c980 20 add_watcher() i=0 2024-05-09T20:12:07.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.618+0000 7f08a874c980 20 add_watcher() i=1 2024-05-09T20:12:07.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.618+0000 7f08a874c980 20 add_watcher() i=2 2024-05-09T20:12:07.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.618+0000 7f08a874c980 20 add_watcher() i=3 2024-05-09T20:12:07.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.618+0000 7f08a874c980 20 add_watcher() i=4 2024-05-09T20:12:07.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.618+0000 7f08a874c980 20 add_watcher() i=5 2024-05-09T20:12:07.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.618+0000 7f08a874c980 20 add_watcher() i=6 2024-05-09T20:12:07.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.622+0000 7f08a874c980 20 add_watcher() i=7 2024-05-09T20:12:07.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.622+0000 7f08a874c980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:07.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.626+0000 7f075cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:07.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.626+0000 7f08a874c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:07.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.626+0000 7f08a874c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:07.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.626+0000 7f08a874c980 5 note: GC not initialized 2024-05-09T20:12:07.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.626+0000 7f06d77fe640 20 reqs_thread_entry: start 2024-05-09T20:12:07.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.626+0000 7f08a874c980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:07.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 Filter name: none 2024-05-09T20:12:07.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f06d57fa640 20 reqs_thread_entry: start 2024-05-09T20:12:07.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:07.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 rados->read ofs=0 len=0 2024-05-09T20:12:07.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:07.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:07.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:12:07.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 INFO: main: starting shard=0 2024-05-09T20:12:07.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:07.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 INFO: main: starting shard=1 2024-05-09T20:12:07.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:07.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 INFO: main: starting shard=2 2024-05-09T20:12:07.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.630+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:07.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 INFO: main: starting shard=3 2024-05-09T20:12:07.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:07.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 INFO: main: starting shard=4 2024-05-09T20:12:07.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:07.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 INFO: main: starting shard=5 2024-05-09T20:12:07.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.634+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:07.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=5, is_truncated=0, marker= 2024-05-09T20:12:07.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: starting shard=6 2024-05-09T20:12:07.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:07.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: starting shard=7 2024-05-09T20:12:07.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:07.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: starting shard=8 2024-05-09T20:12:07.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:07.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: starting shard=9 2024-05-09T20:12:07.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:07.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: starting shard=10 2024-05-09T20:12:07.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:07.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:07.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.638+0000 7f08a874c980 20 INFO: main: done 2024-05-09T20:12:07.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.642+0000 7f075cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:07.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.646+0000 7f08a874c980 20 remove_watcher() i=0 2024-05-09T20:12:07.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.646+0000 7f08a874c980 2 removed watcher, disabling cache 2024-05-09T20:12:07.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.650+0000 7f08a874c980 20 remove_watcher() i=1 2024-05-09T20:12:07.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.654+0000 7f08a874c980 20 remove_watcher() i=2 2024-05-09T20:12:07.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.654+0000 7f08a874c980 20 remove_watcher() i=3 2024-05-09T20:12:07.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.662+0000 7f08a874c980 20 remove_watcher() i=4 2024-05-09T20:12:07.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.662+0000 7f08a874c980 20 remove_watcher() i=5 2024-05-09T20:12:07.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.666+0000 7f08a874c980 20 remove_watcher() i=6 2024-05-09T20:12:07.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:07.666+0000 7f08a874c980 20 remove_watcher() i=7 2024-05-09T20:12:07.909 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:12:07.909 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object e 2024-05-09T20:12:07.945 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:07.946 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:08.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.182+0000 7f38af4f2640 0 --2- 172.21.15.63:0/2136729449 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55eb6eb89c20 0x55eb6ed19e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-09T20:12:08.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.206+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.210+0000 7f38b19de980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.210+0000 7f38b19de980 20 realm 2024-05-09T20:12:08.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.210+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.210+0000 7f38b19de980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.210+0000 7f38b19de980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:08.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.210+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.210+0000 7f38b19de980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.210+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.214+0000 7f38b19de980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:08.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.214+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.214+0000 7f38b19de980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:08.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.214+0000 7f38b19de980 20 searching for the correct realm 2024-05-09T20:12:08.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.214+0000 7f38aecf1640 0 -- 172.21.15.63:0/339260891 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f38a010b040 msgr2=0x7f38a01079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:12:08.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.426+0000 7f38aecf1640 0 -- 172.21.15.63:0/339260891 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f38a00cb070 msgr2=0x7f38a010fad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:08.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:08.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:08.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:08.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:08.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:08.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:08.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:08.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 zone default found 2024-05-09T20:12:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 4 Realm: () 2024-05-09T20:12:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:08.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 zonegroup default 2024-05-09T20:12:08.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.650+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.654+0000 7f38b19de980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.654+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.654+0000 7f38b19de980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.654+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.654+0000 7f38b19de980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.654+0000 7f38b19de980 20 started sync module instance, tier type = 2024-05-09T20:12:08.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.654+0000 7f38b19de980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:08.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 20 add_watcher() i=0 2024-05-09T20:12:08.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 20 add_watcher() i=1 2024-05-09T20:12:08.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 20 add_watcher() i=2 2024-05-09T20:12:08.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 20 add_watcher() i=3 2024-05-09T20:12:08.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 20 add_watcher() i=4 2024-05-09T20:12:08.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 20 add_watcher() i=5 2024-05-09T20:12:08.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 20 add_watcher() i=6 2024-05-09T20:12:08.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 20 add_watcher() i=7 2024-05-09T20:12:08.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.682+0000 7f38b19de980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:08.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f3765fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:08.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f38b19de980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:08.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f38b19de980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:08.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f38b19de980 5 note: GC not initialized 2024-05-09T20:12:08.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f3764fe1640 20 reqs_thread_entry: start 2024-05-09T20:12:08.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f38b19de980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:08.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f38b19de980 20 Filter name: none 2024-05-09T20:12:08.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f36de7fc640 20 reqs_thread_entry: start 2024-05-09T20:12:08.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f38b19de980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:08.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.690+0000 7f38b19de980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:08.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:08.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:12:08.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 INFO: main: starting shard=0 2024-05-09T20:12:08.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:08.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 INFO: main: starting shard=1 2024-05-09T20:12:08.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:08.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-09T20:12:08.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 INFO: main: starting shard=2 2024-05-09T20:12:08.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:08.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 INFO: main: starting shard=3 2024-05-09T20:12:08.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.694+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:08.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 INFO: main: starting shard=4 2024-05-09T20:12:08.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:08.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 INFO: main: starting shard=5 2024-05-09T20:12:08.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:08.695 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.695 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 INFO: main: starting shard=6 2024-05-09T20:12:08.695 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.698+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:08.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: starting shard=7 2024-05-09T20:12:08.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:08.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: starting shard=8 2024-05-09T20:12:08.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:08.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: starting shard=9 2024-05-09T20:12:08.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:08.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: starting shard=10 2024-05-09T20:12:08.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:08.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:08.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.702+0000 7f38b19de980 20 INFO: main: done 2024-05-09T20:12:08.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.706+0000 7f3765fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:08.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.710+0000 7f38b19de980 20 remove_watcher() i=0 2024-05-09T20:12:08.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.710+0000 7f38b19de980 2 removed watcher, disabling cache 2024-05-09T20:12:08.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.710+0000 7f38b19de980 20 remove_watcher() i=1 2024-05-09T20:12:08.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.714+0000 7f38b19de980 20 remove_watcher() i=2 2024-05-09T20:12:08.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.718+0000 7f38b19de980 20 remove_watcher() i=3 2024-05-09T20:12:08.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.718+0000 7f38b19de980 20 remove_watcher() i=4 2024-05-09T20:12:08.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.722+0000 7f38b19de980 20 remove_watcher() i=5 2024-05-09T20:12:08.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.722+0000 7f38b19de980 20 remove_watcher() i=6 2024-05-09T20:12:08.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.722+0000 7f38b19de980 20 remove_watcher() i=7 2024-05-09T20:12:08.889 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:12:08.889 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket check-bucket --object f 2024-05-09T20:12:08.926 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:08.926 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:08.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 20 realm 2024-05-09T20:12:08.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:08.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:08.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.978+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.982+0000 7fab1ed60980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:08.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.982+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:08.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.982+0000 7fab1ed60980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:08.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.982+0000 7fab1ed60980 20 searching for the correct realm 2024-05-09T20:12:08.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:08.986+0000 7fab1c874640 0 -- 172.21.15.63:0/3585005170 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fab140d3020 msgr2=0x7fab140b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:09.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.202+0000 7fab1c874640 0 -- 172.21.15.63:0/3585005170 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fab140d3020 msgr2=0x7fab140b98b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:09.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:09.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:09.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:09.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:09.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:09.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:09.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:09.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:09.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:09.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:09.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:09.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:09.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 zone default found 2024-05-09T20:12:09.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 4 Realm: () 2024-05-09T20:12:09.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:09.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:09.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:09.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 zonegroup default 2024-05-09T20:12:09.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.214+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:09.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.218+0000 7fab1ed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:09.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.218+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:09.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.218+0000 7fab1ed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:09.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.218+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:09.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.218+0000 7fab1ed60980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:09.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.218+0000 7fab1ed60980 20 started sync module instance, tier type = 2024-05-09T20:12:09.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.218+0000 7fab1ed60980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:09.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.242+0000 7fab1ed60980 20 add_watcher() i=0 2024-05-09T20:12:09.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.242+0000 7fab1ed60980 20 add_watcher() i=1 2024-05-09T20:12:09.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.242+0000 7fab1ed60980 20 add_watcher() i=2 2024-05-09T20:12:09.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.242+0000 7fab1ed60980 20 add_watcher() i=3 2024-05-09T20:12:09.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.242+0000 7fab1ed60980 20 add_watcher() i=4 2024-05-09T20:12:09.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.242+0000 7fab1ed60980 20 add_watcher() i=5 2024-05-09T20:12:09.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.242+0000 7fab1ed60980 20 add_watcher() i=6 2024-05-09T20:12:09.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.246+0000 7fab1ed60980 20 add_watcher() i=7 2024-05-09T20:12:09.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.246+0000 7fab1ed60980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:09.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fa9d2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:09.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fab1ed60980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:09.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fab1ed60980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:09.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fab1ed60980 5 note: GC not initialized 2024-05-09T20:12:09.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fa9d1fe3640 20 reqs_thread_entry: start 2024-05-09T20:12:09.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fab1ed60980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:09.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fab1ed60980 20 Filter name: none 2024-05-09T20:12:09.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fa9437fe640 20 reqs_thread_entry: start 2024-05-09T20:12:09.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:09.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fab1ed60980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:09.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.250+0000 7fab1ed60980 20 rados->read ofs=0 len=0 2024-05-09T20:12:09.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:09.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:09.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 INFO: main: max_entries=1000, index=gen=0, max_shards=11 2024-05-09T20:12:09.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 INFO: main: starting shard=0 2024-05-09T20:12:09.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:09.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 INFO: main: starting shard=1 2024-05-09T20:12:09.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:09.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 INFO: main: starting shard=2 2024-05-09T20:12:09.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:09.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 INFO: main: starting shard=3 2024-05-09T20:12:09.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.254+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:09.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: starting shard=4 2024-05-09T20:12:09.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:09.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: starting shard=5 2024-05-09T20:12:09.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:09.254 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.254 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: starting shard=6 2024-05-09T20:12:09.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:09.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: starting shard=7 2024-05-09T20:12:09.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:09.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 INFO: main: starting shard=8 2024-05-09T20:12:09.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.258+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:09.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fab1ed60980 20 INFO: main: starting shard=9 2024-05-09T20:12:09.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:09.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:12:09.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fab1ed60980 20 INFO: main: starting shard=10 2024-05-09T20:12:09.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fab1ed60980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:09.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fab1ed60980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-09T20:12:09.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fab1ed60980 20 INFO: main: done 2024-05-09T20:12:09.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.262+0000 7fa9d2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:09.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.270+0000 7fab1ed60980 20 remove_watcher() i=0 2024-05-09T20:12:09.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.270+0000 7fab1ed60980 2 removed watcher, disabling cache 2024-05-09T20:12:09.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.270+0000 7fab1ed60980 20 remove_watcher() i=1 2024-05-09T20:12:09.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.274+0000 7fab1ed60980 20 remove_watcher() i=2 2024-05-09T20:12:09.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.274+0000 7fab1ed60980 20 remove_watcher() i=3 2024-05-09T20:12:09.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.278+0000 7fab1ed60980 20 remove_watcher() i=4 2024-05-09T20:12:09.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.278+0000 7fab1ed60980 20 remove_watcher() i=5 2024-05-09T20:12:09.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.282+0000 7fab1ed60980 20 remove_watcher() i=6 2024-05-09T20:12:09.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:09.282+0000 7fab1ed60980 20 remove_watcher() i=7 2024-05-09T20:12:09.324 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-05-09T20:12:09.739 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-05-09T20:12:10.158 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check --fix --bucket check-bucket 2024-05-09T20:12:10.182 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:10.182 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:10.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 20 realm 2024-05-09T20:12:10.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:10.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.226+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.230+0000 7f25732e0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:10.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.230+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.230+0000 7f25732e0980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:10.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.230+0000 7f25732e0980 20 searching for the correct realm 2024-05-09T20:12:10.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.234+0000 7f2570df4640 0 -- 172.21.15.63:0/1086988479 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f2568018070 msgr2=0x7f25680637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:12:10.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.450+0000 7f2570df4640 0 -- 172.21.15.63:0/1086988479 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f2568018070 msgr2=0x7f25680637a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:10.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:10.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:10.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:10.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:10.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:10.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:10.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 zone default found 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 4 Realm: () 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 zonegroup default 2024-05-09T20:12:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.462+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.466+0000 7f25732e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.466+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.466+0000 7f25732e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.466+0000 7f25732e0980 20 started sync module instance, tier type = 2024-05-09T20:12:10.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.466+0000 7f25732e0980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:10.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.486+0000 7f25732e0980 20 add_watcher() i=0 2024-05-09T20:12:10.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.486+0000 7f25732e0980 20 add_watcher() i=1 2024-05-09T20:12:10.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.486+0000 7f25732e0980 20 add_watcher() i=2 2024-05-09T20:12:10.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.486+0000 7f25732e0980 20 add_watcher() i=3 2024-05-09T20:12:10.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.486+0000 7f25732e0980 20 add_watcher() i=4 2024-05-09T20:12:10.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.486+0000 7f25732e0980 20 add_watcher() i=5 2024-05-09T20:12:10.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.486+0000 7f25732e0980 20 add_watcher() i=6 2024-05-09T20:12:10.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.490+0000 7f25732e0980 20 add_watcher() i=7 2024-05-09T20:12:10.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.490+0000 7f25732e0980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:10.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f2426fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:10.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:10.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:10.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 5 note: GC not initialized 2024-05-09T20:12:10.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f2425fe3640 20 reqs_thread_entry: start 2024-05-09T20:12:10.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:10.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 20 Filter name: none 2024-05-09T20:12:10.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f23977fe640 20 reqs_thread_entry: start 2024-05-09T20:12:10.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:12:10.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.494+0000 7f25732e0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:12:10.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:12:10.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:12:10.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:12:10.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:12:10.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:12:10.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:12:10.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.498+0000 7f25732e0980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:12:10.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.510+0000 7f2426fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:10.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.514+0000 7f25732e0980 20 remove_watcher() i=0 2024-05-09T20:12:10.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.514+0000 7f25732e0980 2 removed watcher, disabling cache 2024-05-09T20:12:10.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.518+0000 7f25732e0980 20 remove_watcher() i=1 2024-05-09T20:12:10.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.518+0000 7f25732e0980 20 remove_watcher() i=2 2024-05-09T20:12:10.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.522+0000 7f25732e0980 20 remove_watcher() i=3 2024-05-09T20:12:10.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.522+0000 7f25732e0980 20 remove_watcher() i=4 2024-05-09T20:12:10.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.526+0000 7f25732e0980 20 remove_watcher() i=5 2024-05-09T20:12:10.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.526+0000 7f25732e0980 20 remove_watcher() i=6 2024-05-09T20:12:10.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.530+0000 7f25732e0980 20 remove_watcher() i=7 2024-05-09T20:12:10.546 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check unlinked --bucket check-bucket --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-05-09T20:12:10.586 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:10.586 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:10.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 realm 2024-05-09T20:12:10.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:10.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.638+0000 7f8832130980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:10.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.642+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.642+0000 7f8832130980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:10.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.642+0000 7f8832130980 20 searching for the correct realm 2024-05-09T20:12:10.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.642+0000 7f8830445640 0 -- 172.21.15.63:0/2305172614 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f88240e8060 msgr2=0x7f88240ea960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:12:10.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.650+0000 7f882f443640 0 -- 172.21.15.63:0/2305172614 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f8820008040 msgr2=0x7f8820054130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:10.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:10.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:10.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 zone default found 2024-05-09T20:12:10.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 4 Realm: () 2024-05-09T20:12:10.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:10.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:10.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:10.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 zonegroup default 2024-05-09T20:12:10.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:10.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 started sync module instance, tier type = 2024-05-09T20:12:10.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.662+0000 7f8832130980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:10.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.678+0000 7f8832130980 20 add_watcher() i=0 2024-05-09T20:12:10.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.678+0000 7f8832130980 20 add_watcher() i=1 2024-05-09T20:12:10.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.678+0000 7f8832130980 20 add_watcher() i=2 2024-05-09T20:12:10.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.678+0000 7f8832130980 20 add_watcher() i=3 2024-05-09T20:12:10.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.678+0000 7f8832130980 20 add_watcher() i=4 2024-05-09T20:12:10.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.678+0000 7f8832130980 20 add_watcher() i=5 2024-05-09T20:12:10.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.678+0000 7f8832130980 20 add_watcher() i=6 2024-05-09T20:12:10.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f8832130980 20 add_watcher() i=7 2024-05-09T20:12:10.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f8832130980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:10.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f86e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:10.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f8832130980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:10.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f8832130980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:10.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f8832130980 5 note: GC not initialized 2024-05-09T20:12:10.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f86e4fe1640 20 reqs_thread_entry: start 2024-05-09T20:12:10.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f8832130980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:10.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.682+0000 7f8832130980 20 Filter name: none 2024-05-09T20:12:10.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f865e7fc640 20 reqs_thread_entry: start 2024-05-09T20:12:10.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 cache get: name=default.rgw.meta+root+check-bucket : miss 2024-05-09T20:12:10.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:10.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 cache put: name=default.rgw.meta+root+check-bucket info.flags=0x11 2024-05-09T20:12:10.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 adding default.rgw.meta+root+check-bucket to cache LRU end 2024-05-09T20:12:10.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 cache get: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 : miss 2024-05-09T20:12:10.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 20 rados->read ofs=0 len=0 2024-05-09T20:12:10.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:10.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 cache put: name=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 info.flags=0x17 2024-05-09T20:12:10.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 adding default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 to cache LRU end 2024-05-09T20:12:10.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:12:10.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 updating xattr: name=user.rgw.acl bl.length()=179 2024-05-09T20:12:10.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.check-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1 2024-05-09T20:12:10.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.0 2024-05-09T20:12:10.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 1 NOTICE: finished shard 0 (0 entries removed) 2024-05-09T20:12:10.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:10.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.2 2024-05-09T20:12:10.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 1 NOTICE: finished shard 2 (0 entries removed) 2024-05-09T20:12:10.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.686+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.3 2024-05-09T20:12:10.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.690+0000 7f8832130980 1 NOTICE: finished shard 3 (0 entries removed) 2024-05-09T20:12:10.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.690+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.4 2024-05-09T20:12:10.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.690+0000 7f8832130980 1 NOTICE: finished shard 4 (0 entries removed) 2024-05-09T20:12:10.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.690+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:10.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.690+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.6 2024-05-09T20:12:10.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 1 NOTICE: finished shard 6 (0 entries removed) 2024-05-09T20:12:10.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.7 2024-05-09T20:12:10.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 1 NOTICE: finished shard 7 (0 entries removed) 2024-05-09T20:12:10.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:10.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.9 2024-05-09T20:12:10.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 1 NOTICE: finished shard 9 (0 entries removed) 2024-05-09T20:12:10.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:10.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:10.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e13ef60 obj=check-bucket:e state=0x55bb9e3f2a18 s->prefetch_data=0 2024-05-09T20:12:10.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.694+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:10.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e0ee290 obj=check-bucket:d state=0x55bb9e3eb5d8 s->prefetch_data=0 2024-05-09T20:12:10.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:10.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e3f6b20 obj=check-bucket:c state=0x55bb9e3eeeb8 s->prefetch_data=0 2024-05-09T20:12:10.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:10.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e3f78f0 obj=check-bucket:f state=0x55bb9e3eff28 s->prefetch_data=0 2024-05-09T20:12:10.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state: setting s->obj_tag to 13gcsaa3f99gxaajl9v3aceczk3tjbpx 2024-05-09T20:12:10.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state_impl: setting s->olh_tag to 5a44fli9p2v0oiyw0xf64e8ow23sgbh1 2024-05-09T20:12:10.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=5a44fli9p2v0oiyw0xf64e8ow23sgbh1 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state: setting s->obj_tag to 7t2dq29ysosqumnjb2tro9xnxab5yu2w 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state_impl: setting s->olh_tag to 0pn9yu5kiacerq61lhulsdz8eeuvydxc 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=0pn9yu5kiacerq61lhulsdz8eeuvydxc 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state: setting s->obj_tag to 95yvepipalc88h30vtte2ni2ntmnor55 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state_impl: setting s->olh_tag to 11yh2927dxn7n0w9lsi92rr9onfqm3jq 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=11yh2927dxn7n0w9lsi92rr9onfqm3jq 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state: setting s->obj_tag to ty9p6t0hmiytwxn1ye0kim670w2ggf4v 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 get_obj_state_impl: setting s->olh_tag to 16yhl9upyg94h2liunyck24qekp87i1b 2024-05-09T20:12:10.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.698+0000 7f8832130980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=16yhl9upyg94h2liunyck24qekp87i1b 2024-05-09T20:12:10.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.702+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:10.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.702+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:10.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.702+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:10.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.702+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:10.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.702+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:10.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.702+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:10.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.702+0000 7f8832130980 20 olh_log_entry: epoch=3 op=3 key=c[DuJookk-42AqPOJzatHGPRG98UXRB73] 2024-05-09T20:12:10.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e3f6b20 obj=check-bucket:_:DuJookk-42AqPOJzatHGPRG98UXRB73_c state=0x55bb9e48cfe8 s->prefetch_data=0 2024-05-09T20:12:10.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 olh_log_entry: epoch=2 op=2 key=f[] 2024-05-09T20:12:10.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 olh_log_entry: epoch=2 op=3 key=f[0aOdCUWnPLSuxU58e5raHd8ilG.h1dK] 2024-05-09T20:12:10.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e3f78f0 obj=check-bucket:_:0aOdCUWnPLSuxU58e5raHd8ilG.h1dK_f state=0x55bb9e49cf08 s->prefetch_data=0 2024-05-09T20:12:10.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 10 manifest: total_size = 9 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12165714600541447512 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e3f78f0 obj=check-bucket:_:0aOdCUWnPLSuxU58e5raHd8ilG.h1dK_f state=0x55bb9e49cf08 s->prefetch_data=0 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 prepare_atomic_modification: state is not atomic. state=0x55bb9e49cf08 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 10 manifest: total_size = 11 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.5927569553919362082 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e3f6b20 obj=check-bucket:_:DuJookk-42AqPOJzatHGPRG98UXRB73_c state=0x55bb9e48cfe8 s->prefetch_data=0 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 prepare_atomic_modification: state is not atomic. state=0x55bb9e48cfe8 2024-05-09T20:12:10.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:10.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:10.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:10.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 olh_log_entry: epoch=2 op=2 key=e[] 2024-05-09T20:12:10.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 olh_log_entry: epoch=2 op=3 key=e[jhPoKKio17AShsRq..iYs3fL3QfM-eE] 2024-05-09T20:12:10.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e13ef60 obj=check-bucket:_:jhPoKKio17AShsRq..iYs3fL3QfM-eE_e state=0x55bb9e4aa5f8 s->prefetch_data=0 2024-05-09T20:12:10.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 olh_log_entry: epoch=3 op=3 key=d[z6pmRnGYIPE4HXuju7ND2Lp0103Bwv6] 2024-05-09T20:12:10.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e0ee290 obj=check-bucket:_:z6pmRnGYIPE4HXuju7ND2Lp0103Bwv6_d state=0x55bb9e4ad178 s->prefetch_data=0 2024-05-09T20:12:10.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 10 manifest: total_size = 9 2024-05-09T20:12:10.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18308430306232524542 2024-05-09T20:12:10.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e13ef60 obj=check-bucket:_:jhPoKKio17AShsRq..iYs3fL3QfM-eE_e state=0x55bb9e4aa5f8 s->prefetch_data=0 2024-05-09T20:12:10.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 prepare_atomic_modification: state is not atomic. state=0x55bb9e4aa5f8 2024-05-09T20:12:10.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:10.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 10 manifest: total_size = 22 2024-05-09T20:12:10.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11398940155551340149 2024-05-09T20:12:10.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e0ee290 obj=check-bucket:_:z6pmRnGYIPE4HXuju7ND2Lp0103Bwv6_d state=0x55bb9e4ad178 s->prefetch_data=0 2024-05-09T20:12:10.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 prepare_atomic_modification: state is not atomic. state=0x55bb9e4ad178 2024-05-09T20:12:10.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.706+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:10.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.710+0000 7f870e7fc640 20 handle_completion(): completion ok for obj=f[0aOdCUWnPLSuxU58e5raHd8ilG.h1dK] 2024-05-09T20:12:10.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.714+0000 7f870e7fc640 20 handle_completion(): completion ok for obj=c[DuJookk-42AqPOJzatHGPRG98UXRB73] 2024-05-09T20:12:10.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.718+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:10.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.718+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:10.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.718+0000 7f870effd640 20 handle_completion(): completion ok for obj=e[jhPoKKio17AShsRq..iYs3fL3QfM-eE] 2024-05-09T20:12:10.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.718+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e3f78f0 obj=check-bucket:f state=0x55bb9e3eff28 s->prefetch_data=0 2024-05-09T20:12:10.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.718+0000 7f8832130980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=16yhl9upyg94h2liunyck24qekp87i1b 2024-05-09T20:12:10.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.718+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.8 2024-05-09T20:12:10.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.718+0000 7f8832130980 1 NOTICE: finished shard 8 (1 entries removed) 2024-05-09T20:12:10.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.722+0000 7f8832130980 20 get_obj_state: octx=0x55bb9e13ef60 obj=check-bucket:e state=0x55bb9e3f2a18 s->prefetch_data=0 2024-05-09T20:12:10.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.722+0000 7f8832130980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=5a44fli9p2v0oiyw0xf64e8ow23sgbh1 2024-05-09T20:12:10.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.722+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:10.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.722+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:10.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.722+0000 7f870effd640 20 handle_completion(): completion ok for obj=d[z6pmRnGYIPE4HXuju7ND2Lp0103Bwv6] 2024-05-09T20:12:10.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.5 2024-05-09T20:12:10.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 1 NOTICE: finished shard 5 (1 entries removed) 2024-05-09T20:12:10.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:10.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:10.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.10 2024-05-09T20:12:10.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 1 NOTICE: finished shard 10 (1 entries removed) 2024-05-09T20:12:10.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1.1 2024-05-09T20:12:10.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 1 NOTICE: finished shard 1 (1 entries removed) 2024-05-09T20:12:10.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.730+0000 7f8832130980 1 NOTICE: finished all shards (4 entries removed) 2024-05-09T20:12:10.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.734+0000 7f86e5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:10.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.738+0000 7f8832130980 20 remove_watcher() i=0 2024-05-09T20:12:10.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.738+0000 7f8832130980 2 removed watcher, disabling cache 2024-05-09T20:12:10.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.738+0000 7f8832130980 20 remove_watcher() i=1 2024-05-09T20:12:10.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.742+0000 7f8832130980 20 remove_watcher() i=2 2024-05-09T20:12:10.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.742+0000 7f8832130980 20 remove_watcher() i=3 2024-05-09T20:12:10.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.746+0000 7f8832130980 20 remove_watcher() i=4 2024-05-09T20:12:10.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.746+0000 7f8832130980 20 remove_watcher() i=5 2024-05-09T20:12:10.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.750+0000 7f8832130980 20 remove_watcher() i=6 2024-05-09T20:12:10.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:10.750+0000 7f8832130980 20 remove_watcher() i=7 2024-05-09T20:12:11.181 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket check-bucket 2024-05-09T20:12:11.212 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:12:11.212 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:12:11.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.262+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:11.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 realm 2024-05-09T20:12:11.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:11.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:12:11.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:11.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:11.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:12:11.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.266+0000 7fd631e92980 20 searching for the correct realm 2024-05-09T20:12:11.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.270+0000 7fd6301a7640 0 -- 172.21.15.63:0/186843952 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fd624024070 msgr2=0x7fd624056540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:12:11.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.278+0000 7fd6301a7640 0 -- 172.21.15.63:0/186843952 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fd4f80749d0 msgr2=0x7fd4f8076e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:12:11.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:12:11.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:12:11.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:12:11.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:12:11.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:12:11.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:12:11.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:11.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.286+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:12:11.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:12:11.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 zone default found 2024-05-09T20:12:11.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 4 Realm: () 2024-05-09T20:12:11.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:12:11.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:12:11.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:12:11.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 zonegroup default 2024-05-09T20:12:11.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:11.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:11.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:12:11.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 started sync module instance, tier type = 2024-05-09T20:12:11.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.290+0000 7fd631e92980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:12:11.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.310+0000 7fd631e92980 20 add_watcher() i=0 2024-05-09T20:12:11.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.310+0000 7fd631e92980 20 add_watcher() i=1 2024-05-09T20:12:11.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.310+0000 7fd631e92980 20 add_watcher() i=2 2024-05-09T20:12:11.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.310+0000 7fd631e92980 20 add_watcher() i=3 2024-05-09T20:12:11.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.310+0000 7fd631e92980 20 add_watcher() i=4 2024-05-09T20:12:11.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.310+0000 7fd631e92980 20 add_watcher() i=5 2024-05-09T20:12:11.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.310+0000 7fd631e92980 20 add_watcher() i=6 2024-05-09T20:12:11.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.314+0000 7fd631e92980 20 add_watcher() i=7 2024-05-09T20:12:11.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.314+0000 7fd631e92980 2 all 8 watchers are set, enabling cache 2024-05-09T20:12:11.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd4e67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:11.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:12:11.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:12:11.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 5 note: GC not initialized 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd4e57e2640 20 reqs_thread_entry: start 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 init_complete bucket index max shards: 11 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 Filter name: none 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd4de7fc640 20 reqs_thread_entry: start 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:11.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rados_obj.operate() r=0 bl.length=190 2024-05-09T20:12:11.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rados->read ofs=0 len=0 2024-05-09T20:12:11.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 rados_obj.operate() r=0 bl.length=404 2024-05-09T20:12:11.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.318+0000 7fd631e92980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:12:11.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.322+0000 7fd4e67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:12:11.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.326+0000 7fd631e92980 20 remove_watcher() i=0 2024-05-09T20:12:11.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.326+0000 7fd631e92980 2 removed watcher, disabling cache 2024-05-09T20:12:11.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.330+0000 7fd631e92980 20 remove_watcher() i=1 2024-05-09T20:12:11.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.330+0000 7fd631e92980 20 remove_watcher() i=2 2024-05-09T20:12:11.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.334+0000 7fd631e92980 20 remove_watcher() i=3 2024-05-09T20:12:11.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.334+0000 7fd631e92980 20 remove_watcher() i=4 2024-05-09T20:12:11.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.338+0000 7fd631e92980 20 remove_watcher() i=5 2024-05-09T20:12:11.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.338+0000 7fd631e92980 20 remove_watcher() i=6 2024-05-09T20:12:11.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:12:11.338+0000 7fd631e92980 20 remove_watcher() i=7 2024-05-09T20:12:11.349 INFO:tasks.workunit.client.0.smithi063.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-05-09T20:12:11.349 INFO:tasks.workunit.client.0.smithi063.stderr:Deleting bucket check-bucket 2024-05-09T20:12:11.949 INFO:tasks.workunit.client.0.smithi063.stderr:Completed bucket check tests 2024-05-09T20:12:11.950 INFO:tasks.workunit.client.0.smithi063.stdout:connected to http://localhost:80 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ deactivate 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ export PATH 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ unset _OLD_VIRTUAL_PATH 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' -n '' ']' 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ hash -r 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' -n '' ']' 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ unset VIRTUAL_ENV 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-05-09T20:12:12.051 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' '!' '' = nondestructive ']' 2024-05-09T20:12:12.052 INFO:tasks.workunit.client.0.smithi063.stderr:+ unset -f deactivate 2024-05-09T20:12:12.052 INFO:tasks.workunit.client.0.smithi063.stderr:+ echo OK. 2024-05-09T20:12:12.052 INFO:tasks.workunit.client.0.smithi063.stdout:OK. 2024-05-09T20:12:12.053 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:12:12.053 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:12:12.110 INFO:tasks.workunit:Stopping ['rgw/run-bucket-check.sh'] on client.0... 2024-05-09T20:12:12.110 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T20:12:12.548 DEBUG:teuthology.parallel:result is None 2024-05-09T20:12:12.549 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:12:12.562 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:12:12.562 DEBUG:teuthology.orchestra.run.smithi063:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:12:12.620 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:12:12.621 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T20:12:12.631 INFO:tasks.workunit:Pulling workunits from ref 86872c6b9201173c508ab909aaa65bdb12239dd6 2024-05-09T20:12:12.631 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T20:12:12.631 DEBUG:teuthology.orchestra.run.smithi063:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:12:12.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:12:12.669 INFO:teuthology.orchestra.run.smithi063.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-09T20:12:12.669 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:12:12.721 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-09T20:12:12.721 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-09T20:12:12.769 INFO:tasks.workunit:timeout=3h 2024-05-09T20:12:12.769 INFO:tasks.workunit:cleanup=True 2024-05-09T20:12:12.770 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 86872c6b9201173c508ab909aaa65bdb12239dd6 2024-05-09T20:12:12.823 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T20:13:15.926 INFO:tasks.workunit.client.0.smithi063.stderr:Note: switching to '86872c6b9201173c508ab909aaa65bdb12239dd6'. 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr: git switch -c 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr:Or undo this operation with: 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:13:15.927 INFO:tasks.workunit.client.0.smithi063.stderr: git switch - 2024-05-09T20:13:15.928 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:13:15.928 INFO:tasks.workunit.client.0.smithi063.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T20:13:15.928 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:13:15.928 INFO:tasks.workunit.client.0.smithi063.stderr:HEAD is now at 86872c6b920 d4n: Add more in-depth config descriptions and improve naming 2024-05-09T20:13:15.931 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T20:13:15.981 INFO:tasks.workunit.client.0.smithi063.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T20:13:15.984 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T20:13:15.985 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T20:13:16.037 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T20:13:16.097 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T20:13:16.138 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T20:13:16.140 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T20:13:16.140 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T20:13:16.186 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T20:13:16.189 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:13:16.189 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T20:13:16.235 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2024-05-09T20:13:16.237 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2024-05-09T20:13:16.237 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2024-05-09T20:13:16.299 INFO:tasks.workunit.client.0.smithi063.stdout:Running main() from gmock_main.cc 2024-05-09T20:13:16.299 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 11 tests from 1 test suite. 2024-05-09T20:13:16.299 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:13:16.299 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 11 tests from ClsLock 2024-05-09T20:13:16.299 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestMultiLocking 2024-05-09T20:13:17.643 INFO:tasks.workunit.client.0.smithi063.stdout:lock: mylock 2024-05-09T20:13:17.644 INFO:tasks.workunit.client.0.smithi063.stdout: lock_type: exclusive 2024-05-09T20:13:17.644 INFO:tasks.workunit.client.0.smithi063.stdout: tag: 2024-05-09T20:13:17.644 INFO:tasks.workunit.client.0.smithi063.stdout: lockers: 2024-05-09T20:13:17.644 INFO:tasks.workunit.client.0.smithi063.stdout: client.5139 expiration=2024-05-09T20:15:17.643862+0000 addr=v1:172.21.15.63:0/1187067703 cookie= 2024-05-09T20:13:17.655 INFO:tasks.workunit.client.0.smithi063.stdout:lock: mylock 2024-05-09T20:13:17.655 INFO:tasks.workunit.client.0.smithi063.stdout: lock_type: shared 2024-05-09T20:13:17.655 INFO:tasks.workunit.client.0.smithi063.stdout: tag: 2024-05-09T20:13:17.655 INFO:tasks.workunit.client.0.smithi063.stdout: lockers: 2024-05-09T20:13:17.656 INFO:tasks.workunit.client.0.smithi063.stdout: client.5139 expiration=2024-05-09T20:15:17.661541+0000 addr=v1:172.21.15.63:0/1187067703 cookie= 2024-05-09T20:13:17.656 INFO:tasks.workunit.client.0.smithi063.stdout: client.5148 expiration=0.000000 addr=v1:172.21.15.63:0/409777640 cookie= 2024-05-09T20:13:17.659 INFO:tasks.workunit.client.0.smithi063.stdout:lock: mylock 2024-05-09T20:13:17.659 INFO:tasks.workunit.client.0.smithi063.stdout: lock_type: shared 2024-05-09T20:13:17.659 INFO:tasks.workunit.client.0.smithi063.stdout: tag: 2024-05-09T20:13:17.659 INFO:tasks.workunit.client.0.smithi063.stdout: lockers: 2024-05-09T20:13:17.659 INFO:tasks.workunit.client.0.smithi063.stdout: client.5148 expiration=0.000000 addr=v1:172.21.15.63:0/409777640 cookie= 2024-05-09T20:13:18.478 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestMultiLocking (2179 ms) 2024-05-09T20:13:18.478 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestMeta 2024-05-09T20:13:20.766 INFO:tasks.workunit.client.0.smithi063.stdout:lock: mylock 2024-05-09T20:13:20.767 INFO:tasks.workunit.client.0.smithi063.stdout: lock_type: shared 2024-05-09T20:13:20.767 INFO:tasks.workunit.client.0.smithi063.stdout: tag: 2024-05-09T20:13:20.767 INFO:tasks.workunit.client.0.smithi063.stdout: lockers: 2024-05-09T20:13:20.767 INFO:tasks.workunit.client.0.smithi063.stdout: client.4970 expiration=0.000000 addr=v1:172.21.15.63:0/2329022466 cookie= 2024-05-09T20:13:20.773 INFO:tasks.workunit.client.0.smithi063.stdout:lock: mylock 2024-05-09T20:13:20.773 INFO:tasks.workunit.client.0.smithi063.stdout: lock_type: exclusive 2024-05-09T20:13:20.773 INFO:tasks.workunit.client.0.smithi063.stdout: tag: new_tag 2024-05-09T20:13:20.773 INFO:tasks.workunit.client.0.smithi063.stdout: lockers: 2024-05-09T20:13:20.773 INFO:tasks.workunit.client.0.smithi063.stdout: client.4964 expiration=0.000000 addr=v1:172.21.15.63:0/412159572 cookie= 2024-05-09T20:13:21.522 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestMeta (3044 ms) 2024-05-09T20:13:21.522 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestCookie 2024-05-09T20:13:23.569 INFO:tasks.workunit.client.0.smithi063.stdout:lock: mylock 2024-05-09T20:13:23.569 INFO:tasks.workunit.client.0.smithi063.stdout: lock_type: exclusive 2024-05-09T20:13:23.569 INFO:tasks.workunit.client.0.smithi063.stdout: tag: 2024-05-09T20:13:23.569 INFO:tasks.workunit.client.0.smithi063.stdout: lockers: 2024-05-09T20:13:23.574 INFO:tasks.workunit.client.0.smithi063.stdout:lock: mylock 2024-05-09T20:13:23.574 INFO:tasks.workunit.client.0.smithi063.stdout: lock_type: shared 2024-05-09T20:13:23.574 INFO:tasks.workunit.client.0.smithi063.stdout: tag: 2024-05-09T20:13:23.574 INFO:tasks.workunit.client.0.smithi063.stdout: lockers: 2024-05-09T20:13:23.574 INFO:tasks.workunit.client.0.smithi063.stdout: client.5166 expiration=0.000000 addr=v1:172.21.15.63:0/3355417787 cookie= 2024-05-09T20:13:23.574 INFO:tasks.workunit.client.0.smithi063.stdout: client.5166 expiration=0.000000 addr=v1:172.21.15.63:0/3355417787 cookie=new cookie 2024-05-09T20:13:24.563 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestCookie (3041 ms) 2024-05-09T20:13:24.563 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestMultipleLocks 2024-05-09T20:13:27.609 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestMultipleLocks (3046 ms) 2024-05-09T20:13:27.609 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestLockDuration 2024-05-09T20:13:35.526 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestLockDuration (7917 ms) 2024-05-09T20:13:35.527 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestAssertLocked 2024-05-09T20:13:38.558 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestAssertLocked (3033 ms) 2024-05-09T20:13:38.559 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestSetCookie 2024-05-09T20:13:41.571 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestSetCookie (3011 ms) 2024-05-09T20:13:41.572 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestRenew 2024-05-09T20:14:01.763 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestRenew (20187 ms) 2024-05-09T20:14:01.764 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2024-05-09T20:14:07.894 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (6137 ms) 2024-05-09T20:14:07.894 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2024-05-09T20:14:15.535 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (7636 ms) 2024-05-09T20:14:15.535 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2024-05-09T20:14:23.537 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8004 ms) 2024-05-09T20:14:23.538 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 11 tests from ClsLock (67236 ms total) 2024-05-09T20:14:23.538 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:14:23.538 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:14:23.538 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 11 tests from 1 test suite ran. (67236 ms total) 2024-05-09T20:14:23.538 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 11 tests. 2024-05-09T20:14:23.540 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:14:23.541 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:14:23.597 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2024-05-09T20:14:23.598 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2024-05-09T20:14:23.599 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2024-05-09T20:14:23.663 INFO:tasks.workunit.client.0.smithi063.stdout:Running main() from gmock_main.cc 2024-05-09T20:14:23.663 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 4 tests from 1 test suite. 2024-05-09T20:14:23.663 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:14:23.664 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 4 tests from cls_log 2024-05-09T20:14:23.664 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_log.test_log_add_same_time 2024-05-09T20:14:26.542 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_log.test_log_add_same_time (2878 ms) 2024-05-09T20:14:26.542 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_log.test_log_add_different_time 2024-05-09T20:14:29.607 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_log.test_log_add_different_time (3066 ms) 2024-05-09T20:14:29.608 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_log.trim_by_time 2024-05-09T20:14:32.622 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_log.trim_by_time (3015 ms) 2024-05-09T20:14:32.622 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_log.trim_by_marker 2024-05-09T20:14:35.667 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_log.trim_by_marker (3045 ms) 2024-05-09T20:14:35.667 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 4 tests from cls_log (12004 ms total) 2024-05-09T20:14:35.668 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:14:35.668 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:14:35.668 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 4 tests from 1 test suite ran. (12004 ms total) 2024-05-09T20:14:35.668 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 4 tests. 2024-05-09T20:14:35.678 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:14:35.678 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:14:35.737 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2024-05-09T20:14:35.739 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2024-05-09T20:14:35.739 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2024-05-09T20:14:35.813 INFO:tasks.workunit.client.0.smithi063.stdout:Running main() from gmock_main.cc 2024-05-09T20:14:35.813 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 10 tests from 1 test suite. 2024-05-09T20:14:35.813 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:14:35.813 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 10 tests from cls_refcount 2024-05-09T20:14:35.813 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.test_implicit 2024-05-09T20:14:38.702 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.test_implicit (2888 ms) 2024-05-09T20:14:38.702 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.test_implicit_idempotent 2024-05-09T20:14:41.769 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.test_implicit_idempotent (3068 ms) 2024-05-09T20:14:41.770 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.test_put_snap 2024-05-09T20:14:50.539 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.test_put_snap (8770 ms) 2024-05-09T20:14:50.539 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.test_explicit 2024-05-09T20:14:53.603 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.test_explicit (3064 ms) 2024-05-09T20:14:53.603 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.set 2024-05-09T20:14:56.617 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.set (3013 ms) 2024-05-09T20:14:56.617 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.test_implicit_ec 2024-05-09T20:15:03.664 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.test_implicit_ec (7046 ms) 2024-05-09T20:15:03.664 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.test_implicit_idempotent_ec 2024-05-09T20:15:10.736 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.test_implicit_idempotent_ec (7073 ms) 2024-05-09T20:15:10.737 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.test_put_snap_ec 2024-05-09T20:15:22.862 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.test_put_snap_ec (12125 ms) 2024-05-09T20:15:22.862 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.test_explicit_ec 2024-05-09T20:15:29.958 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.test_explicit_ec (7097 ms) 2024-05-09T20:15:29.958 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_refcount.set_ec 2024-05-09T20:15:36.991 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_refcount.set_ec (7033 ms) 2024-05-09T20:15:36.991 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 10 tests from cls_refcount (61178 ms total) 2024-05-09T20:15:36.991 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:15:36.991 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:15:36.991 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 10 tests from 1 test suite ran. (61178 ms total) 2024-05-09T20:15:36.991 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 10 tests. 2024-05-09T20:15:36.995 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:15:36.995 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:15:37.055 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2024-05-09T20:15:37.056 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2024-05-09T20:15:37.057 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2024-05-09T20:15:37.121 INFO:tasks.workunit.client.0.smithi063.stdout:Running main() from gmock_main.cc 2024-05-09T20:15:37.121 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 16 tests from 1 test suite. 2024-05-09T20:15:37.121 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:15:37.121 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 16 tests from cls_rgw 2024-05-09T20:15:39.052 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.index_basic 2024-05-09T20:15:39.134 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.index_basic (82 ms) 2024-05-09T20:15:39.134 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2024-05-09T20:15:39.220 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (87 ms) 2024-05-09T20:15:39.221 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.index_remove_object 2024-05-09T20:15:39.330 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.index_remove_object (108 ms) 2024-05-09T20:15:39.330 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.index_suggest 2024-05-09T20:15:41.855 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.index_suggest (2526 ms) 2024-05-09T20:15:41.855 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.index_suggest_complete 2024-05-09T20:15:41.870 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.index_suggest_complete (15 ms) 2024-05-09T20:15:41.870 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.index_list 2024-05-09T20:15:41.922 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.index_list (52 ms) 2024-05-09T20:15:41.922 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.index_list_delimited 2024-05-09T20:16:33.880 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.index_list_delimited (51958 ms) 2024-05-09T20:16:33.880 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.bi_list 2024-05-09T20:16:34.033 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.bi_list (153 ms) 2024-05-09T20:16:34.033 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.gc_set 2024-05-09T20:16:34.053 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.gc_set (21 ms) 2024-05-09T20:16:34.053 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.gc_list 2024-05-09T20:16:34.073 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.gc_list (20 ms) 2024-05-09T20:16:34.073 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.gc_defer 2024-05-09T20:16:41.617 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.gc_defer (7543 ms) 2024-05-09T20:16:41.617 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.usage_basic 2024-05-09T20:16:41.840 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.usage_basic (224 ms) 2024-05-09T20:16:41.841 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2024-05-09T20:16:41.844 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.usage_clear_no_obj (3 ms) 2024-05-09T20:16:41.844 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.usage_clear 2024-05-09T20:16:42.061 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.usage_clear (217 ms) 2024-05-09T20:16:42.061 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.bi_log_trim 2024-05-09T20:16:42.189 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.bi_log_trim (128 ms) 2024-05-09T20:16:42.189 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw.index_racing_removes 2024-05-09T20:16:42.227 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw.index_racing_removes (34 ms) 2024-05-09T20:16:42.647 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 16 tests from cls_rgw (63171 ms total) 2024-05-09T20:16:42.647 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:16:42.647 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:16:42.647 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 16 tests from 1 test suite ran. (65527 ms total) 2024-05-09T20:16:42.647 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 16 tests. 2024-05-09T20:16:42.656 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:16:42.656 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:16:42.712 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_gc.sh on client.0... 2024-05-09T20:16:42.713 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2024-05-09T20:16:42.714 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2024-05-09T20:16:42.778 INFO:tasks.workunit.client.0.smithi063.stdout:Running main() from gmock_main.cc 2024-05-09T20:16:42.778 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 9 tests from 1 test suite. 2024-05-09T20:16:42.778 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:16:42.778 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 9 tests from cls_rgw_gc 2024-05-09T20:16:42.778 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.init 2024-05-09T20:16:44.655 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.init (1878 ms) 2024-05-09T20:16:44.655 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2024-05-09T20:16:44.691 INFO:tasks.workunit.client.0.smithi063.stderr:[ ] list info tag = chain-0 2024-05-09T20:16:44.691 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (35 ms) 2024-05-09T20:16:44.691 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2024-05-09T20:16:44.703 INFO:tasks.workunit.client.0.smithi063.stderr:[ ] next_marker is: = 0/4263 2024-05-09T20:16:44.704 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (14 ms) 2024-05-09T20:16:44.704 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2024-05-09T20:16:44.721 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (17 ms) 2024-05-09T20:16:44.722 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2024-05-09T20:16:45.168 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (447 ms) 2024-05-09T20:16:45.168 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2024-05-09T20:16:45.411 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (243 ms) 2024-05-09T20:16:45.411 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2024-05-09T20:16:45.429 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (17 ms) 2024-05-09T20:16:45.429 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2024-05-09T20:16:45.445 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (17 ms) 2024-05-09T20:16:45.446 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_gc.finalize 2024-05-09T20:16:45.694 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_gc.finalize (249 ms) 2024-05-09T20:16:45.694 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 9 tests from cls_rgw_gc (2917 ms total) 2024-05-09T20:16:45.694 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:16:45.695 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:16:45.695 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 9 tests from 1 test suite ran. (2917 ms total) 2024-05-09T20:16:45.695 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 9 tests. 2024-05-09T20:16:45.698 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:16:45.699 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:16:45.755 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw_stats.sh on client.0... 2024-05-09T20:16:45.756 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2024-05-09T20:16:45.756 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2024-05-09T20:16:45.837 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:16:45.837 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:16:45.837 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 1 test from 1 test suite. 2024-05-09T20:16:45.837 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:16:47.700 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 1 test from cls_rgw_stats 2024-05-09T20:16:47.701 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] cls_rgw_stats.simulate 2024-05-09T20:16:47.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.740+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=VM57QA2AZL5T type=del size=2024409 2024-05-09T20:16:47.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.744+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~B44RDH4406UK.0 size=1048576 2024-05-09T20:16:47.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.748+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~B44RDH4406UK.1 size=1048576 2024-05-09T20:16:47.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.748+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~B44RDH4406UK.2 size=475526 2024-05-09T20:16:47.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.748+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=B44RDH4406UK tag=TV4P7Z3SBIB0 type=write size=3621254 2024-05-09T20:16:47.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.752+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=ZW7P6PTONYXF type=del size=2987089 2024-05-09T20:16:47.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.752+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=KAR361MQZF8R type=del size=1979408 2024-05-09T20:16:47.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.760+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~J31YSV4IU1OH.0 size=1048576 2024-05-09T20:16:47.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.760+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~J31YSV4IU1OH.1 size=278465 2024-05-09T20:16:47.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.760+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=J31YSV4IU1OH tag=K2C9A8ZGNL70 type=write size=2375617 2024-05-09T20:16:47.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.764+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=OJPP338PRW40 type=write size=905888 2024-05-09T20:16:47.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.768+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~TK1P1JASR170.0 size=1048576 2024-05-09T20:16:47.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.772+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~TK1P1JASR170.1 size=1048576 2024-05-09T20:16:47.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.772+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~TK1P1JASR170.2 size=859267 2024-05-09T20:16:47.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.772+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=TK1P1JASR170 tag=OV33GF9GSHFQ type=write size=4004995 2024-05-09T20:16:47.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.776+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=Y3ONHOEWB7TK type=del size=1541506 2024-05-09T20:16:47.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~Q1NYUNAI1KZN.0 size=1048576 2024-05-09T20:16:47.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~Q1NYUNAI1KZN.1 size=102582 2024-05-09T20:16:47.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.784+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=Q1NYUNAI1KZN tag=X7ZCON5QLNLV type=write size=2199734 2024-05-09T20:16:47.779 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.788+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~H490R1AEYU79.0 size=1048576 2024-05-09T20:16:47.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.788+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~H490R1AEYU79.1 size=22749 2024-05-09T20:16:47.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.788+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=H490R1AEYU79 tag=20C9FOSL2CEL type=write size=2119901 2024-05-09T20:16:47.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.792+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=83G12EGAGYDX type=del size=329101 2024-05-09T20:16:47.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.796+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~7DICJQBAPSE2.0 size=505647 2024-05-09T20:16:47.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.796+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=7DICJQBAPSE2 tag=ZI4TCSW10VVP type=write size=1554223 2024-05-09T20:16:47.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.800+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=SQG815ZSI5EB type=del size=664599 2024-05-09T20:16:47.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.804+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=A91KU0XPRD6L type=del size=1233540 2024-05-09T20:16:47.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.804+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=R4E129RJ2OT6 type=write size=484769 2024-05-09T20:16:47.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.808+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=GM41NPJCRMTR type=write size=912051 2024-05-09T20:16:47.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.812+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=VM57QA2AZL5T type=del 2024-05-09T20:16:47.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.812+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=XUF4E1G7SQHT type=del size=4184736 2024-05-09T20:16:47.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.816+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~B44RDH4406UK.0 size=1048576 2024-05-09T20:16:47.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.820+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~B44RDH4406UK.1 size=1048576 2024-05-09T20:16:47.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.820+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~B44RDH4406UK.2 size=475526 2024-05-09T20:16:47.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.828+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=B44RDH4406UK tag=TV4P7Z3SBIB0 type=write size=3621254 2024-05-09T20:16:47.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.828+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=MSGUA7XPDOQW type=write size=576893 2024-05-09T20:16:47.828 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.836+0000 7fdc882e58c0 1 < suggested update operation key=P2DNA4DGI54P tag=ZW7P6PTONYXF type=del size=2987089 2024-05-09T20:16:47.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.840+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=ZLBOACN9CWYW type=del size=1417525 2024-05-09T20:16:47.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.844+0000 7fdc882e58c0 1 < canceled operation key=JS90W6PIREMD tag=KAR361MQZF8R type=del size=1979408 2024-05-09T20:16:47.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.852+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~ODNNI6WXW04N.0 size=1048576 2024-05-09T20:16:47.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.852+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~ODNNI6WXW04N.1 size=1036739 2024-05-09T20:16:47.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.852+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=ODNNI6WXW04N tag=STK5Q1337ZPX type=write size=3133891 2024-05-09T20:16:47.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.856+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~J31YSV4IU1OH.0 size=1048576 2024-05-09T20:16:47.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.860+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~J31YSV4IU1OH.1 size=278465 2024-05-09T20:16:47.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.864+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=J31YSV4IU1OH tag=K2C9A8ZGNL70 type=write size=2375617 2024-05-09T20:16:47.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.868+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=E86B1XCW6700 type=write size=698275 2024-05-09T20:16:47.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.872+0000 7fdc882e58c0 1 < completed write operation key=5H2GWM9LQELZ tag=OJPP338PRW40 type=write size=905888 2024-05-09T20:16:47.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.876+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~6DZG9K5O06AG.0 size=1048576 2024-05-09T20:16:47.871 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.880+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~6DZG9K5O06AG.1 size=1048576 2024-05-09T20:16:47.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.880+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~6DZG9K5O06AG.2 size=229201 2024-05-09T20:16:47.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.880+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=6DZG9K5O06AG tag=50DWX7U44SJN type=write size=3374929 2024-05-09T20:16:47.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.884+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~TK1P1JASR170.0 size=1048576 2024-05-09T20:16:47.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.888+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~TK1P1JASR170.1 size=1048576 2024-05-09T20:16:47.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.892+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~TK1P1JASR170.2 size=859267 2024-05-09T20:16:47.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.896+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=TK1P1JASR170 tag=OV33GF9GSHFQ type=write size=4004995 2024-05-09T20:16:47.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~85XOSJLRU2JU.0 size=944811 2024-05-09T20:16:47.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.900+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=85XOSJLRU2JU tag=VHS2TV4BZWWV type=write size=1993387 2024-05-09T20:16:47.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.908+0000 7fdc882e58c0 1 < suggested update operation key=IZPPFLWVPRKO tag=Y3ONHOEWB7TK type=del size=1541506 2024-05-09T20:16:47.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.912+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=4W20SE3C1SLI type=del size=2744705 2024-05-09T20:16:47.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.916+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~Q1NYUNAI1KZN.0 size=1048576 2024-05-09T20:16:47.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.920+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~Q1NYUNAI1KZN.1 size=102582 2024-05-09T20:16:47.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.924+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=Q1NYUNAI1KZN tag=X7ZCON5QLNLV type=write size=2199734 2024-05-09T20:16:47.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.928+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~BUEXNA49BNKZ.0 size=964270 2024-05-09T20:16:47.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.928+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=BUEXNA49BNKZ tag=57XV00YKXAJF type=write size=2012846 2024-05-09T20:16:47.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.932+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~H490R1AEYU79.0 size=1048576 2024-05-09T20:16:47.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.936+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~H490R1AEYU79.1 size=22749 2024-05-09T20:16:47.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.940+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=H490R1AEYU79 tag=20C9FOSL2CEL type=write size=2119901 2024-05-09T20:16:47.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.944+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~MNXAUCMKHDTX.0 size=1042996 2024-05-09T20:16:47.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.944+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=MNXAUCMKHDTX tag=V441IPDXJDWV type=write size=2091572 2024-05-09T20:16:47.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.948+0000 7fdc882e58c0 -1 < failed to complete operation key=IZPPFLWVPRKO tag=83G12EGAGYDX type=del size=329101: (22) Invalid argument 2024-05-09T20:16:47.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.952+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=3MURPN3BDS1H type=write size=276413 2024-05-09T20:16:47.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.956+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~7DICJQBAPSE2.0 size=505647 2024-05-09T20:16:47.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.960+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=7DICJQBAPSE2 tag=ZI4TCSW10VVP type=write size=1554223 2024-05-09T20:16:47.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.968+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~6U94OU4GTZW9.0 size=193272 2024-05-09T20:16:47.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.968+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=6U94OU4GTZW9 tag=9RGT3JBYXGUU type=write size=1241848 2024-05-09T20:16:47.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.972+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=SQG815ZSI5EB type=del 2024-05-09T20:16:47.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.972+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=H0EPH9VNELFQ type=del size=2562528 2024-05-09T20:16:47.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.980+0000 7fdc882e58c0 1 < suggested remove operation key=WC5YH0OU51WX tag=A91KU0XPRD6L type=del 2024-05-09T20:16:47.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.984+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=8P7KFG3XAP3W type=del size=3712873 2024-05-09T20:16:47.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.984+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=R4E129RJ2OT6 type=write size=484769 2024-05-09T20:16:47.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.992+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~U3NF4GO1W3ET.0 size=1048576 2024-05-09T20:16:47.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.992+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~U3NF4GO1W3ET.1 size=632181 2024-05-09T20:16:47.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.992+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=U3NF4GO1W3ET tag=IFTBQ9A4MD62 type=write size=2729333 2024-05-09T20:16:47.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:47.996+0000 7fdc882e58c0 1 < completed write operation key=5H2GWM9LQELZ tag=GM41NPJCRMTR type=write size=912051 2024-05-09T20:16:47.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.000+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=9J9NMDZWK8IJ type=write size=571129 2024-05-09T20:16:47.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.004+0000 7fdc882e58c0 1 < suggested update operation key=54R9XPO62J0M tag=XUF4E1G7SQHT type=del size=4184736 2024-05-09T20:16:48.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~RF5DOXA6TAXS.0 size=1048576 2024-05-09T20:16:48.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.020+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~RF5DOXA6TAXS.1 size=1048576 2024-05-09T20:16:48.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.020+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~RF5DOXA6TAXS.2 size=164548 2024-05-09T20:16:48.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.020+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=RF5DOXA6TAXS tag=YQ139BRWAJEL type=write size=3310276 2024-05-09T20:16:48.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.024+0000 7fdc882e58c0 1 < completed write operation key=G0DX1F257RLD tag=MSGUA7XPDOQW type=write size=576893 2024-05-09T20:16:48.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.028+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=LEGC069KYZII type=del size=935239 2024-05-09T20:16:48.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.036+0000 7fdc882e58c0 1 < suggested remove operation key=JS90W6PIREMD tag=ZLBOACN9CWYW type=del 2024-05-09T20:16:48.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.036+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=MC0LIDKHIVMH type=del size=172519 2024-05-09T20:16:48.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.040+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~ODNNI6WXW04N.0 size=1048576 2024-05-09T20:16:48.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.044+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~ODNNI6WXW04N.1 size=1036739 2024-05-09T20:16:48.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.048+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=ODNNI6WXW04N tag=STK5Q1337ZPX type=write size=3133891 2024-05-09T20:16:48.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.056+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=G1FZOWGMZMI9 type=del size=3545444 2024-05-09T20:16:48.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.060+0000 7fdc882e58c0 1 < suggested remove operation key=O07UHAIJZF0R tag=E86B1XCW6700 type=write 2024-05-09T20:16:48.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.064+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=22PY7DJ9OR6J type=del size=535997 2024-05-09T20:16:48.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.068+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~6DZG9K5O06AG.0 size=1048576 2024-05-09T20:16:48.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.072+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~6DZG9K5O06AG.1 size=1048576 2024-05-09T20:16:48.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.072+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~6DZG9K5O06AG.2 size=229201 2024-05-09T20:16:48.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.080+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=6DZG9K5O06AG tag=50DWX7U44SJN type=write size=3374929 2024-05-09T20:16:48.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.084+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=SBJPII6Y8SHO type=del size=1593028 2024-05-09T20:16:48.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.084+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~85XOSJLRU2JU.0 size=944811 2024-05-09T20:16:48.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.092+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=85XOSJLRU2JU tag=VHS2TV4BZWWV type=write size=1993387 2024-05-09T20:16:48.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.096+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=ARXHR1XAFIRN type=del size=461310 2024-05-09T20:16:48.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.100+0000 7fdc882e58c0 -1 < failed to complete operation key=WC5YH0OU51WX tag=4W20SE3C1SLI type=del size=2744705: (22) Invalid argument 2024-05-09T20:16:48.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.104+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=8GQ8N2A8A7IL type=del size=4018667 2024-05-09T20:16:48.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.104+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~BUEXNA49BNKZ.0 size=964270 2024-05-09T20:16:48.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.112+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=BUEXNA49BNKZ tag=57XV00YKXAJF type=write size=2012846 2024-05-09T20:16:48.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.116+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=WVPAZFKCIXLT type=del size=1783599 2024-05-09T20:16:48.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.116+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~MNXAUCMKHDTX.0 size=1042996 2024-05-09T20:16:48.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.124+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=MNXAUCMKHDTX tag=V441IPDXJDWV type=write size=2091572 2024-05-09T20:16:48.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.128+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~ATN24HU4GGZM.0 size=1048576 2024-05-09T20:16:48.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.132+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~ATN24HU4GGZM.1 size=86378 2024-05-09T20:16:48.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.132+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=ATN24HU4GGZM tag=0F9WAU08GVDB type=write size=2183530 2024-05-09T20:16:48.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.136+0000 7fdc882e58c0 1 < completed write operation key=NYHR6GTPQYZS tag=3MURPN3BDS1H type=write size=276413 2024-05-09T20:16:48.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.136+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=ZX3ATLKJL132 type=write size=158162 2024-05-09T20:16:48.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.140+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~6U94OU4GTZW9.0 size=193272 2024-05-09T20:16:48.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.148+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=6U94OU4GTZW9 tag=9RGT3JBYXGUU type=write size=1241848 2024-05-09T20:16:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.152+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~XU6E9Z5T0TBS.0 size=221609 2024-05-09T20:16:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.152+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=XU6E9Z5T0TBS tag=V7YZRBQRQX1T type=write size=1270185 2024-05-09T20:16:48.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.156+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=H0EPH9VNELFQ type=del 2024-05-09T20:16:48.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.160+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=1U93ZDHHLI5S type=del size=2287110 2024-05-09T20:16:48.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.160+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=8P7KFG3XAP3W type=del 2024-05-09T20:16:48.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.168+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~B2WO377N6L9A.0 size=1048576 2024-05-09T20:16:48.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.168+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~B2WO377N6L9A.1 size=966580 2024-05-09T20:16:48.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.168+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=B2WO377N6L9A tag=1XJNETW9PUVW type=write size=3063732 2024-05-09T20:16:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.172+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~U3NF4GO1W3ET.0 size=1048576 2024-05-09T20:16:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.176+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~U3NF4GO1W3ET.1 size=632181 2024-05-09T20:16:48.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.180+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=U3NF4GO1W3ET tag=IFTBQ9A4MD62 type=write size=2729333 2024-05-09T20:16:48.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.188+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~565VEAHM4Y0E.0 size=1048576 2024-05-09T20:16:48.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.188+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~565VEAHM4Y0E.1 size=1048576 2024-05-09T20:16:48.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.192+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~565VEAHM4Y0E.2 size=840696 2024-05-09T20:16:48.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.192+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=565VEAHM4Y0E tag=2864SBO6N640 type=write size=3986424 2024-05-09T20:16:48.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.196+0000 7fdc882e58c0 1 < completed write operation key=EK78CNKUHNKO tag=9J9NMDZWK8IJ type=write size=571129 2024-05-09T20:16:48.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.200+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~5KLHTH0S4U0Z.0 size=186198 2024-05-09T20:16:48.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.200+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=5KLHTH0S4U0Z tag=LADOSTR3PM10 type=write size=1234774 2024-05-09T20:16:48.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.204+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~RF5DOXA6TAXS.0 size=1048576 2024-05-09T20:16:48.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.204+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~RF5DOXA6TAXS.1 size=1048576 2024-05-09T20:16:48.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.208+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~RF5DOXA6TAXS.2 size=164548 2024-05-09T20:16:48.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.212+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=RF5DOXA6TAXS tag=YQ139BRWAJEL type=write size=3310276 2024-05-09T20:16:48.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.216+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=YVJ7RBDPY6VS type=del size=303031 2024-05-09T20:16:48.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.220+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=LEGC069KYZII type=del 2024-05-09T20:16:48.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.224+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=21J8K0RDTKGC type=del size=2647383 2024-05-09T20:16:48.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.228+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=MC0LIDKHIVMH type=del 2024-05-09T20:16:48.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.228+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=1XRGZA3713SD type=del size=3503445 2024-05-09T20:16:48.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.236+0000 7fdc882e58c0 1 < suggested remove operation key=BS0T1OQOBS67 tag=G1FZOWGMZMI9 type=del 2024-05-09T20:16:48.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.240+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=JR6RCL3JF56J type=del size=2990040 2024-05-09T20:16:48.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.244+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=22PY7DJ9OR6J type=del 2024-05-09T20:16:48.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.244+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=V8L7POK3VQLQ type=del size=1420727 2024-05-09T20:16:48.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.248+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=SBJPII6Y8SHO type=del 2024-05-09T20:16:48.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.252+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=L2KXBV91NPPP type=del size=1493340 2024-05-09T20:16:48.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.256+0000 7fdc882e58c0 1 < suggested remove operation key=QM53IENH9CX4 tag=ARXHR1XAFIRN type=del 2024-05-09T20:16:48.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.260+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=A6JEJQ0ZXB68 type=del size=1377706 2024-05-09T20:16:48.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.264+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=8GQ8N2A8A7IL type=del 2024-05-09T20:16:48.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.268+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~NF3DMJUTIBZX.0 size=459878 2024-05-09T20:16:48.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.268+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=NF3DMJUTIBZX tag=0BWQN0HRL0ZF type=write size=1508454 2024-05-09T20:16:48.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.272+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=WVPAZFKCIXLT type=del 2024-05-09T20:16:48.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.280+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~2L6OK211CF4W.0 size=1048576 2024-05-09T20:16:48.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.284+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~2L6OK211CF4W.1 size=166206 2024-05-09T20:16:48.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.284+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=2L6OK211CF4W tag=HYPFOYOBOPRP type=write size=2263358 2024-05-09T20:16:48.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.284+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~ATN24HU4GGZM.0 size=1048576 2024-05-09T20:16:48.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.288+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~ATN24HU4GGZM.1 size=86378 2024-05-09T20:16:48.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.296+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=ATN24HU4GGZM tag=0F9WAU08GVDB type=write size=2183530 2024-05-09T20:16:48.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.300+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~GPQZWOMU3UC9.0 size=1048576 2024-05-09T20:16:48.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.304+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~GPQZWOMU3UC9.1 size=104833 2024-05-09T20:16:48.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.304+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=GPQZWOMU3UC9 tag=1FSU7NMKU74P type=write size=2201985 2024-05-09T20:16:48.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.308+0000 7fdc882e58c0 -1 < failed to complete operation key=QM53IENH9CX4 tag=ZX3ATLKJL132 type=write size=158162: (22) Invalid argument 2024-05-09T20:16:48.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.312+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=QZ0GTBFQCQU8 type=del size=3286956 2024-05-09T20:16:48.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.312+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~XU6E9Z5T0TBS.0 size=221609 2024-05-09T20:16:48.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.320+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=XU6E9Z5T0TBS tag=V7YZRBQRQX1T type=write size=1270185 2024-05-09T20:16:48.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.324+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~AL7WJY7G0FVD.0 size=1048576 2024-05-09T20:16:48.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.328+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~AL7WJY7G0FVD.1 size=1048576 2024-05-09T20:16:48.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.332+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~AL7WJY7G0FVD.2 size=516004 2024-05-09T20:16:48.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.332+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=AL7WJY7G0FVD tag=MR6JXB4V5JP2 type=write size=3661732 2024-05-09T20:16:48.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.340+0000 7fdc882e58c0 1 < suggested remove operation key=J2Q86HKRMAKE tag=1U93ZDHHLI5S type=del 2024-05-09T20:16:48.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.344+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=6PG0YM1INTGZ type=write size=63890 2024-05-09T20:16:48.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.344+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~B2WO377N6L9A.0 size=1048576 2024-05-09T20:16:48.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.348+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~B2WO377N6L9A.1 size=966580 2024-05-09T20:16:48.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.352+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=B2WO377N6L9A tag=1XJNETW9PUVW type=write size=3063732 2024-05-09T20:16:48.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.364+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~R5OSISMM8ZM6.0 size=1048576 2024-05-09T20:16:48.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.364+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~R5OSISMM8ZM6.1 size=667011 2024-05-09T20:16:48.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.364+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=R5OSISMM8ZM6 tag=66LE8TTNA649 type=write size=2764163 2024-05-09T20:16:48.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.368+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~565VEAHM4Y0E.0 size=1048576 2024-05-09T20:16:48.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.372+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~565VEAHM4Y0E.1 size=1048576 2024-05-09T20:16:48.366 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.376+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~565VEAHM4Y0E.2 size=840696 2024-05-09T20:16:48.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.380+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=565VEAHM4Y0E tag=2864SBO6N640 type=write size=3986424 2024-05-09T20:16:48.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.384+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=W9SMJ5UQLSZ1 type=del size=892242 2024-05-09T20:16:48.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.388+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~5KLHTH0S4U0Z.0 size=186198 2024-05-09T20:16:48.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.396+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=5KLHTH0S4U0Z tag=LADOSTR3PM10 type=write size=1234774 2024-05-09T20:16:48.390 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.400+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=5Z933RZG2CPU type=del size=2515620 2024-05-09T20:16:48.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.400+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=YVJ7RBDPY6VS type=del 2024-05-09T20:16:48.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.408+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~BQRQG3GUCCJB.0 size=1048576 2024-05-09T20:16:48.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.412+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~BQRQG3GUCCJB.1 size=324735 2024-05-09T20:16:48.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.412+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=BQRQG3GUCCJB tag=0QN8ZJU7TZ43 type=write size=2421887 2024-05-09T20:16:48.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.416+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=21J8K0RDTKGC type=del 2024-05-09T20:16:48.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.420+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=VNDMVLU0OY8W type=del size=2943789 2024-05-09T20:16:48.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.424+0000 7fdc882e58c0 1 < canceled operation key=5NEOR8OOFAFZ tag=1XRGZA3713SD type=del size=3503445 2024-05-09T20:16:48.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.424+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=L4M6798JBAXD type=write size=786766 2024-05-09T20:16:48.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.428+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=JR6RCL3JF56J type=del 2024-05-09T20:16:48.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.432+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=SSQLK91V5F6U type=del size=1700486 2024-05-09T20:16:48.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.436+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=V8L7POK3VQLQ type=del 2024-05-09T20:16:48.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.440+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~UIA6L9DMBC8J.0 size=1048576 2024-05-09T20:16:48.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.444+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~UIA6L9DMBC8J.1 size=1000852 2024-05-09T20:16:48.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.444+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=UIA6L9DMBC8J tag=UVXJ02FSGOZE type=write size=3098004 2024-05-09T20:16:48.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.448+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=L2KXBV91NPPP type=del 2024-05-09T20:16:48.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.448+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=VAY5MTYMB9T5 type=write size=229131 2024-05-09T20:16:48.443 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.452+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=A6JEJQ0ZXB68 type=del 2024-05-09T20:16:48.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.456+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=0G08RFBHC2BF type=del size=3634877 2024-05-09T20:16:48.450 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.456+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~NF3DMJUTIBZX.0 size=459878 2024-05-09T20:16:48.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.464+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=NF3DMJUTIBZX tag=0BWQN0HRL0ZF type=write size=1508454 2024-05-09T20:16:48.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.468+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~UM3396EECFIT.0 size=1048576 2024-05-09T20:16:48.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.468+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~UM3396EECFIT.1 size=769465 2024-05-09T20:16:48.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.468+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=UM3396EECFIT tag=BUW6BP8WEPMR type=write size=2866617 2024-05-09T20:16:48.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.472+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~2L6OK211CF4W.0 size=1048576 2024-05-09T20:16:48.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.476+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~2L6OK211CF4W.1 size=166206 2024-05-09T20:16:48.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.480+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=2L6OK211CF4W tag=HYPFOYOBOPRP type=write size=2263358 2024-05-09T20:16:48.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~BO78LG6L1EPV.0 size=1048576 2024-05-09T20:16:48.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.488+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~BO78LG6L1EPV.1 size=1048576 2024-05-09T20:16:48.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.492+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~BO78LG6L1EPV.2 size=445076 2024-05-09T20:16:48.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.492+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=BO78LG6L1EPV tag=BWSTMAWCGMX9 type=write size=3590804 2024-05-09T20:16:48.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.496+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~GPQZWOMU3UC9.0 size=1048576 2024-05-09T20:16:48.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.496+0000 7fdc882e58c0 1 < canceled part key=_multipart_5UMJFJRKUPKR.2~GPQZWOMU3UC9.1 size=104833 2024-05-09T20:16:48.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.500+0000 7fdc882e58c0 1 < canceled multipart upload key=5UMJFJRKUPKR upload=GPQZWOMU3UC9 tag=1FSU7NMKU74P type=write size=2201985 2024-05-09T20:16:48.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.504+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=AIF3JDU4XNNT type=del size=1129429 2024-05-09T20:16:48.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.504+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=QZ0GTBFQCQU8 type=del 2024-05-09T20:16:48.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.512+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~LMFMOYC25G6P.0 size=1048576 2024-05-09T20:16:48.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.512+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~LMFMOYC25G6P.1 size=697658 2024-05-09T20:16:48.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.512+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=LMFMOYC25G6P tag=BBKK5VAFZH1N type=write size=2794810 2024-05-09T20:16:48.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.516+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~AL7WJY7G0FVD.0 size=1048576 2024-05-09T20:16:48.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.516+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~AL7WJY7G0FVD.1 size=1048576 2024-05-09T20:16:48.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.520+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~AL7WJY7G0FVD.2 size=516004 2024-05-09T20:16:48.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.524+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=AL7WJY7G0FVD tag=MR6JXB4V5JP2 type=write size=3661732 2024-05-09T20:16:48.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.532+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~GH0QDEVOGQHL.0 size=1048576 2024-05-09T20:16:48.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.532+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~GH0QDEVOGQHL.1 size=1048576 2024-05-09T20:16:48.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.536+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~GH0QDEVOGQHL.2 size=567778 2024-05-09T20:16:48.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.536+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=GH0QDEVOGQHL tag=NW7OE09ZZRIU type=write size=3713506 2024-05-09T20:16:48.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.540+0000 7fdc882e58c0 1 < completed write operation key=346HV1XQKCOZ tag=6PG0YM1INTGZ type=write size=63890 2024-05-09T20:16:48.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.540+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=325YSM7BCP1D type=write size=143541 2024-05-09T20:16:48.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.540+0000 7fdc882e58c0 1 < canceled part key=_multipart_54R9XPO62J0M.2~R5OSISMM8ZM6.0 size=1048576 2024-05-09T20:16:48.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.544+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~R5OSISMM8ZM6.1 size=667011 2024-05-09T20:16:48.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.552+0000 7fdc882e58c0 1 < canceled multipart upload key=54R9XPO62J0M upload=R5OSISMM8ZM6 tag=66LE8TTNA649 type=write size=2764163 2024-05-09T20:16:48.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.552+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=FOCAT5TKN5T7 type=del size=744120 2024-05-09T20:16:48.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.556+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=W9SMJ5UQLSZ1 type=del 2024-05-09T20:16:48.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.560+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=Z7EY9F6P3AHO type=del size=3889727 2024-05-09T20:16:48.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.564+0000 7fdc882e58c0 1 < canceled operation key=T5EL7X0P450L tag=5Z933RZG2CPU type=del size=2515620 2024-05-09T20:16:48.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.568+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~OP291CH0MGSU.0 size=139636 2024-05-09T20:16:48.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.568+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=OP291CH0MGSU tag=16I2HQDHA185 type=write size=1188212 2024-05-09T20:16:48.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.572+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~BQRQG3GUCCJB.0 size=1048576 2024-05-09T20:16:48.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.572+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~BQRQG3GUCCJB.1 size=324735 2024-05-09T20:16:48.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.580+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=BQRQG3GUCCJB tag=0QN8ZJU7TZ43 type=write size=2421887 2024-05-09T20:16:48.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.584+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~RXH2R6GQLDDP.0 size=1048576 2024-05-09T20:16:48.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.584+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~RXH2R6GQLDDP.1 size=1048576 2024-05-09T20:16:48.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~RXH2R6GQLDDP.2 size=989674 2024-05-09T20:16:48.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.588+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=RXH2R6GQLDDP tag=YS43QCZ6D69S type=write size=4135402 2024-05-09T20:16:48.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.592+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=VNDMVLU0OY8W type=del 2024-05-09T20:16:48.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.596+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=HX3XBDQ2VERZ type=del size=989965 2024-05-09T20:16:48.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.604+0000 7fdc882e58c0 1 < suggested update operation key=OGVA2778MJWU tag=L4M6798JBAXD type=write size=786766 2024-05-09T20:16:48.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.608+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=2J0JRYVSX1R5 type=del size=1842689 2024-05-09T20:16:48.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.612+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=SSQLK91V5F6U type=del 2024-05-09T20:16:48.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.616+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=BT6KZY7QLXQ0 type=del size=3462170 2024-05-09T20:16:48.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.616+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~UIA6L9DMBC8J.0 size=1048576 2024-05-09T20:16:48.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.620+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~UIA6L9DMBC8J.1 size=1000852 2024-05-09T20:16:48.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.624+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=UIA6L9DMBC8J tag=UVXJ02FSGOZE type=write size=3098004 2024-05-09T20:16:48.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.632+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~2B3JFWPG5YG1.0 size=397713 2024-05-09T20:16:48.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.632+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=2B3JFWPG5YG1 tag=9Z3XE5TQ22PF type=write size=1446289 2024-05-09T20:16:48.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.640+0000 7fdc882e58c0 1 < suggested update operation key=IZPPFLWVPRKO tag=VAY5MTYMB9T5 type=write size=229131 2024-05-09T20:16:48.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.644+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=KIE3EZRGN6Y3 type=del size=1466892 2024-05-09T20:16:48.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.644+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=0G08RFBHC2BF type=del 2024-05-09T20:16:48.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.652+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~9K8X7LXVR8I9.0 size=1048576 2024-05-09T20:16:48.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.656+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~9K8X7LXVR8I9.1 size=704095 2024-05-09T20:16:48.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.656+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=9K8X7LXVR8I9 tag=N1QL763GFEEW type=write size=2801247 2024-05-09T20:16:48.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.660+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~UM3396EECFIT.0 size=1048576 2024-05-09T20:16:48.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.664+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~UM3396EECFIT.1 size=769465 2024-05-09T20:16:48.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.668+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=UM3396EECFIT tag=BUW6BP8WEPMR type=write size=2866617 2024-05-09T20:16:48.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.672+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~CJB7QJ5FK3A7.0 size=364928 2024-05-09T20:16:48.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.672+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=CJB7QJ5FK3A7 tag=YFX4C5V55DQZ type=write size=1413504 2024-05-09T20:16:48.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.676+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~BO78LG6L1EPV.0 size=1048576 2024-05-09T20:16:48.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.680+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~BO78LG6L1EPV.1 size=1048576 2024-05-09T20:16:48.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.680+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~BO78LG6L1EPV.2 size=445076 2024-05-09T20:16:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.688+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=BO78LG6L1EPV tag=BWSTMAWCGMX9 type=write size=3590804 2024-05-09T20:16:48.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.692+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~VU7368MUIB3K.0 size=1048576 2024-05-09T20:16:48.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.696+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~VU7368MUIB3K.1 size=721294 2024-05-09T20:16:48.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.696+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=VU7368MUIB3K tag=YP8QOGFFXJIL type=write size=2818446 2024-05-09T20:16:48.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.708+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=AIF3JDU4XNNT type=del 2024-05-09T20:16:48.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.716+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~BZ02IEGWB93A.0 size=389063 2024-05-09T20:16:48.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.716+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=BZ02IEGWB93A tag=P42XOQ3ET6RL type=write size=1437639 2024-05-09T20:16:48.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.720+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~LMFMOYC25G6P.0 size=1048576 2024-05-09T20:16:48.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.720+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~LMFMOYC25G6P.1 size=697658 2024-05-09T20:16:48.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.724+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=LMFMOYC25G6P tag=BBKK5VAFZH1N type=write size=2794810 2024-05-09T20:16:48.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.732+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~YCK60HHAE8XG.0 size=1048576 2024-05-09T20:16:48.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.732+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~YCK60HHAE8XG.1 size=1048576 2024-05-09T20:16:48.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~YCK60HHAE8XG.2 size=101409 2024-05-09T20:16:48.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.736+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=YCK60HHAE8XG tag=1F7OBQ03HDUG type=write size=3247137 2024-05-09T20:16:48.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.736+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~GH0QDEVOGQHL.0 size=1048576 2024-05-09T20:16:48.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.736+0000 7fdc882e58c0 1 < canceled part key=_multipart_P2DNA4DGI54P.2~GH0QDEVOGQHL.1 size=1048576 2024-05-09T20:16:48.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.740+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~GH0QDEVOGQHL.2 size=567778 2024-05-09T20:16:48.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.744+0000 7fdc882e58c0 1 < canceled multipart upload key=P2DNA4DGI54P upload=GH0QDEVOGQHL tag=NW7OE09ZZRIU type=write size=3713506 2024-05-09T20:16:48.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.744+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=W3CEQQZ8HIN0 type=del size=252759 2024-05-09T20:16:48.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.748+0000 7fdc882e58c0 1 < completed write operation key=NBAECV9L85UK tag=325YSM7BCP1D type=write size=143541 2024-05-09T20:16:48.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.752+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=24SR6F5K6ZE4 type=del size=63125 2024-05-09T20:16:48.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.752+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=FOCAT5TKN5T7 type=del 2024-05-09T20:16:48.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.756+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=QRULIOEFXAQ1 type=write size=359023 2024-05-09T20:16:48.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.756+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=Z7EY9F6P3AHO type=del 2024-05-09T20:16:48.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.760+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=XFC7S7HJ7IAF type=del size=4081420 2024-05-09T20:16:48.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.764+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~OP291CH0MGSU.0 size=139636 2024-05-09T20:16:48.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.768+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=OP291CH0MGSU tag=16I2HQDHA185 type=write size=1188212 2024-05-09T20:16:48.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.772+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=P2EF2MO18CMG type=del size=2027202 2024-05-09T20:16:48.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.772+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~RXH2R6GQLDDP.0 size=1048576 2024-05-09T20:16:48.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.776+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~RXH2R6GQLDDP.1 size=1048576 2024-05-09T20:16:48.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.776+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~RXH2R6GQLDDP.2 size=989674 2024-05-09T20:16:48.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.780+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=RXH2R6GQLDDP tag=YS43QCZ6D69S type=write size=4135402 2024-05-09T20:16:48.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.784+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=QHI1NH38XK2T type=del size=245991 2024-05-09T20:16:48.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.784+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=HX3XBDQ2VERZ type=del 2024-05-09T20:16:48.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.792+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~M3DNP23PJNUP.0 size=1048576 2024-05-09T20:16:48.786 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.796+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~M3DNP23PJNUP.1 size=968898 2024-05-09T20:16:48.786 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.796+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=M3DNP23PJNUP tag=0D2P2XREHTMN type=write size=3066050 2024-05-09T20:16:48.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.796+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=2J0JRYVSX1R5 type=del 2024-05-09T20:16:48.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.800+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=CJMVVWO3N308 type=del size=3856626 2024-05-09T20:16:48.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.800+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=BT6KZY7QLXQ0 type=del 2024-05-09T20:16:48.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.804+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=771RNCCP2Q6P type=write size=986309 2024-05-09T20:16:48.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.808+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~2B3JFWPG5YG1.0 size=397713 2024-05-09T20:16:48.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.812+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=2B3JFWPG5YG1 tag=9Z3XE5TQ22PF type=write size=1446289 2024-05-09T20:16:48.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.812+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=IR1OGOT6WYJR type=write size=404058 2024-05-09T20:16:48.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.816+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=KIE3EZRGN6Y3 type=del 2024-05-09T20:16:48.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.820+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=PM04FG7ZYYIS type=write size=782640 2024-05-09T20:16:48.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.820+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~9K8X7LXVR8I9.0 size=1048576 2024-05-09T20:16:48.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.824+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~9K8X7LXVR8I9.1 size=704095 2024-05-09T20:16:48.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.828+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=9K8X7LXVR8I9 tag=N1QL763GFEEW type=write size=2801247 2024-05-09T20:16:48.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.832+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~Q8EEHVFZ190F.0 size=1048576 2024-05-09T20:16:48.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.832+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~Q8EEHVFZ190F.1 size=802632 2024-05-09T20:16:48.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.832+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=Q8EEHVFZ190F tag=6DG82UI7Y34Z type=write size=2899784 2024-05-09T20:16:48.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.836+0000 7fdc882e58c0 1 < canceled part key=_multipart_FNB9B18LIA8L.2~CJB7QJ5FK3A7.0 size=364928 2024-05-09T20:16:48.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.840+0000 7fdc882e58c0 1 < canceled multipart upload key=FNB9B18LIA8L upload=CJB7QJ5FK3A7 tag=YFX4C5V55DQZ type=write size=1413504 2024-05-09T20:16:48.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.840+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=D0EKKN5BAN45 type=del size=2065314 2024-05-09T20:16:48.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.844+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~VU7368MUIB3K.0 size=1048576 2024-05-09T20:16:48.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.848+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~VU7368MUIB3K.1 size=721294 2024-05-09T20:16:48.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.852+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=VU7368MUIB3K tag=YP8QOGFFXJIL type=write size=2818446 2024-05-09T20:16:48.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.856+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=0G4U5JVWXB1O type=del size=2440703 2024-05-09T20:16:48.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.856+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~BZ02IEGWB93A.0 size=389063 2024-05-09T20:16:48.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.860+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=BZ02IEGWB93A tag=P42XOQ3ET6RL type=write size=1437639 2024-05-09T20:16:48.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.864+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=K73PVEJ191BR type=write size=85095 2024-05-09T20:16:48.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.864+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~YCK60HHAE8XG.0 size=1048576 2024-05-09T20:16:48.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.868+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~YCK60HHAE8XG.1 size=1048576 2024-05-09T20:16:48.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.868+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~YCK60HHAE8XG.2 size=101409 2024-05-09T20:16:48.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.872+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=YCK60HHAE8XG tag=1F7OBQ03HDUG type=write size=3247137 2024-05-09T20:16:48.870 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.880+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~5U6F18VOOMGF.0 size=1048576 2024-05-09T20:16:48.872 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.880+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~5U6F18VOOMGF.1 size=1048576 2024-05-09T20:16:48.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.884+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~5U6F18VOOMGF.2 size=928061 2024-05-09T20:16:48.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.884+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=5U6F18VOOMGF tag=XYCEINUASSZZ type=write size=4073789 2024-05-09T20:16:48.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.888+0000 7fdc882e58c0 1 < suggested update operation key=QM53IENH9CX4 tag=W3CEQQZ8HIN0 type=del size=252759 2024-05-09T20:16:48.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.892+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~41MLIW93HBFY.0 size=1048576 2024-05-09T20:16:48.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.896+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~41MLIW93HBFY.1 size=1048576 2024-05-09T20:16:48.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~41MLIW93HBFY.2 size=294529 2024-05-09T20:16:48.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.900+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=41MLIW93HBFY tag=VM8KIG376HT2 type=write size=3440257 2024-05-09T20:16:48.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.900+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=24SR6F5K6ZE4 type=del 2024-05-09T20:16:48.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.904+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=0MM812Y4VDR2 type=del size=1674269 2024-05-09T20:16:48.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.908+0000 7fdc882e58c0 1 < completed write operation key=P2DNA4DGI54P tag=QRULIOEFXAQ1 type=write size=359023 2024-05-09T20:16:48.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.908+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=4P8FGUWC2Q51 type=del size=3138427 2024-05-09T20:16:48.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.912+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=XFC7S7HJ7IAF type=del 2024-05-09T20:16:48.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.916+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=9T7N593GYYV2 type=del size=3636616 2024-05-09T20:16:48.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.920+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=P2EF2MO18CMG type=del 2024-05-09T20:16:48.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.920+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=Q8S0IKN8VDZK type=del size=1428886 2024-05-09T20:16:48.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.924+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=QHI1NH38XK2T type=del 2024-05-09T20:16:48.919 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.928+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=JYBRFSBHG3CS type=del size=2513122 2024-05-09T20:16:48.922 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.932+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~M3DNP23PJNUP.0 size=1048576 2024-05-09T20:16:48.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.932+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~M3DNP23PJNUP.1 size=968898 2024-05-09T20:16:48.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.936+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=M3DNP23PJNUP tag=0D2P2XREHTMN type=write size=3066050 2024-05-09T20:16:48.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.940+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=AVI4RJK5N94G type=del size=3802902 2024-05-09T20:16:48.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.944+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=CJMVVWO3N308 type=del 2024-05-09T20:16:48.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.948+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=GDH0Q3ZHGH9D type=del size=145751 2024-05-09T20:16:48.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.948+0000 7fdc882e58c0 1 < completed write operation key=CNV7NHP27JZ7 tag=771RNCCP2Q6P type=write size=986309 2024-05-09T20:16:48.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.952+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=OAE7VX0632UE type=del size=2627289 2024-05-09T20:16:48.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.956+0000 7fdc882e58c0 1 < suggested remove operation key=G0DX1F257RLD tag=IR1OGOT6WYJR type=write 2024-05-09T20:16:48.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.964+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~72XAJ2EFIDOI.0 size=1048576 2024-05-09T20:16:48.957 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.964+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~72XAJ2EFIDOI.1 size=1048576 2024-05-09T20:16:48.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.968+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~72XAJ2EFIDOI.2 size=76444 2024-05-09T20:16:48.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.968+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=72XAJ2EFIDOI tag=31WQPMLMQE0S type=write size=3222172 2024-05-09T20:16:48.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.972+0000 7fdc882e58c0 1 < completed write operation key=HPK2T0629B1P tag=PM04FG7ZYYIS type=write size=782640 2024-05-09T20:16:48.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.976+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~3CXU41UMGDFY.0 size=1048576 2024-05-09T20:16:48.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~3CXU41UMGDFY.1 size=1048576 2024-05-09T20:16:48.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~3CXU41UMGDFY.2 size=138704 2024-05-09T20:16:48.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.980+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=3CXU41UMGDFY tag=04UETA0279TC type=write size=3284432 2024-05-09T20:16:48.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.984+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~Q8EEHVFZ190F.0 size=1048576 2024-05-09T20:16:48.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.988+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~Q8EEHVFZ190F.1 size=802632 2024-05-09T20:16:48.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.992+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=Q8EEHVFZ190F tag=6DG82UI7Y34Z type=write size=2899784 2024-05-09T20:16:48.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:48.996+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~QEQC7163K49F.0 size=1048576 2024-05-09T20:16:48.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.000+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~QEQC7163K49F.1 size=1048576 2024-05-09T20:16:48.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.004+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~QEQC7163K49F.2 size=13086 2024-05-09T20:16:48.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.004+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=QEQC7163K49F tag=KNOMD4T9D9M5 type=write size=3158814 2024-05-09T20:16:48.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.004+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=D0EKKN5BAN45 type=del 2024-05-09T20:16:49.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.008+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=TUWJEN1CUWTQ type=del size=2994550 2024-05-09T20:16:49.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.012+0000 7fdc882e58c0 1 < canceled operation key=8FFZZDURUWM1 tag=0G4U5JVWXB1O type=del size=2440703 2024-05-09T20:16:49.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~OOPY9G5LXMJP.0 size=1048576 2024-05-09T20:16:49.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.020+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~OOPY9G5LXMJP.1 size=1048576 2024-05-09T20:16:49.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.020+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~OOPY9G5LXMJP.2 size=30364 2024-05-09T20:16:49.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.020+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=OOPY9G5LXMJP tag=30OMZXF7W6A1 type=write size=3176092 2024-05-09T20:16:49.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.024+0000 7fdc882e58c0 1 < canceled operation key=SNPNT5JFJOUS tag=K73PVEJ191BR type=write size=85095 2024-05-09T20:16:49.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.028+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~M6QAJFT3QOP3.0 size=1048576 2024-05-09T20:16:49.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.032+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~M6QAJFT3QOP3.1 size=202361 2024-05-09T20:16:49.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.032+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=M6QAJFT3QOP3 tag=KSSLP9TDE5UZ type=write size=2299513 2024-05-09T20:16:49.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.032+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~5U6F18VOOMGF.0 size=1048576 2024-05-09T20:16:49.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.036+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~5U6F18VOOMGF.1 size=1048576 2024-05-09T20:16:49.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.036+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~5U6F18VOOMGF.2 size=928061 2024-05-09T20:16:49.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.044+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=5U6F18VOOMGF tag=XYCEINUASSZZ type=write size=4073789 2024-05-09T20:16:49.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.044+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=ZPEBHOZ7B81M type=write size=233146 2024-05-09T20:16:49.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.048+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~41MLIW93HBFY.0 size=1048576 2024-05-09T20:16:49.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.052+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~41MLIW93HBFY.1 size=1048576 2024-05-09T20:16:49.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.052+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~41MLIW93HBFY.2 size=294529 2024-05-09T20:16:49.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.056+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=41MLIW93HBFY tag=VM8KIG376HT2 type=write size=3440257 2024-05-09T20:16:49.052 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.060+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=HLGJU2E2DUCI type=del size=3825227 2024-05-09T20:16:49.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.064+0000 7fdc882e58c0 1 < suggested update operation key=WC5YH0OU51WX tag=0MM812Y4VDR2 type=del size=1674269 2024-05-09T20:16:49.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.072+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~8HPSZ9O260OY.0 size=1048576 2024-05-09T20:16:49.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.072+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~8HPSZ9O260OY.1 size=1048576 2024-05-09T20:16:49.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.076+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~8HPSZ9O260OY.2 size=1007326 2024-05-09T20:16:49.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.076+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=8HPSZ9O260OY tag=H2DDDSMGUT5I type=write size=4153054 2024-05-09T20:16:49.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.080+0000 7fdc882e58c0 1 < suggested remove operation key=SPXHCWZICCWY tag=4P8FGUWC2Q51 type=del 2024-05-09T20:16:49.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.084+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=HITPQU9BY627 type=del size=2231634 2024-05-09T20:16:49.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.084+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=9T7N593GYYV2 type=del 2024-05-09T20:16:49.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.088+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=OX3ZKNRJDM2S type=del size=2963425 2024-05-09T20:16:49.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.092+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=Q8S0IKN8VDZK type=del 2024-05-09T20:16:49.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.096+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=G3U4S9DDZVHW type=del size=3610094 2024-05-09T20:16:49.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.096+0000 7fdc882e58c0 -1 < failed to complete operation key=G0DX1F257RLD tag=JYBRFSBHG3CS type=del size=2513122: (22) Invalid argument 2024-05-09T20:16:49.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.100+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=3G5MN1IMIQI9 type=del size=3784379 2024-05-09T20:16:49.095 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.104+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=AVI4RJK5N94G type=del 2024-05-09T20:16:49.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.108+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=34NAKQAP3BYG type=del size=2092896 2024-05-09T20:16:49.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.108+0000 7fdc882e58c0 1 < canceled operation key=EIMPFFK4I6NZ tag=GDH0Q3ZHGH9D type=del size=145751 2024-05-09T20:16:49.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.112+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=WBR118HFFUHZ type=del size=2589783 2024-05-09T20:16:49.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.120+0000 7fdc882e58c0 1 < suggested remove operation key=OGVA2778MJWU tag=OAE7VX0632UE type=del 2024-05-09T20:16:49.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.120+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=08HFZYJLY4PW type=del size=1996770 2024-05-09T20:16:49.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.124+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~72XAJ2EFIDOI.0 size=1048576 2024-05-09T20:16:49.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.128+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~72XAJ2EFIDOI.1 size=1048576 2024-05-09T20:16:49.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.132+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~72XAJ2EFIDOI.2 size=76444 2024-05-09T20:16:49.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.136+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=72XAJ2EFIDOI tag=31WQPMLMQE0S type=write size=3222172 2024-05-09T20:16:49.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.140+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=IMT4S7OX1DVY type=del size=1572889 2024-05-09T20:16:49.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.144+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~3CXU41UMGDFY.0 size=1048576 2024-05-09T20:16:49.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.144+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~3CXU41UMGDFY.1 size=1048576 2024-05-09T20:16:49.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.148+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~3CXU41UMGDFY.2 size=138704 2024-05-09T20:16:49.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.156+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=3CXU41UMGDFY tag=04UETA0279TC type=write size=3284432 2024-05-09T20:16:49.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.160+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=8E4GPEG9H7L2 type=del size=3049942 2024-05-09T20:16:49.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.164+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~QEQC7163K49F.0 size=1048576 2024-05-09T20:16:49.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.164+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~QEQC7163K49F.1 size=1048576 2024-05-09T20:16:49.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.168+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~QEQC7163K49F.2 size=13086 2024-05-09T20:16:49.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.172+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=QEQC7163K49F tag=KNOMD4T9D9M5 type=write size=3158814 2024-05-09T20:16:49.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.176+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=HVSI3WFGFFXR type=del size=3961316 2024-05-09T20:16:49.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.180+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=TUWJEN1CUWTQ type=del 2024-05-09T20:16:49.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.184+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=YF70NG4DE4HS type=write size=265701 2024-05-09T20:16:49.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.188+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~OOPY9G5LXMJP.0 size=1048576 2024-05-09T20:16:49.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.192+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~OOPY9G5LXMJP.1 size=1048576 2024-05-09T20:16:49.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.196+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~OOPY9G5LXMJP.2 size=30364 2024-05-09T20:16:49.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.204+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=OOPY9G5LXMJP tag=30OMZXF7W6A1 type=write size=3176092 2024-05-09T20:16:49.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.208+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=AXFCI0KSP7E0 type=write size=227099 2024-05-09T20:16:49.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.212+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~M6QAJFT3QOP3.0 size=1048576 2024-05-09T20:16:49.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.212+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~M6QAJFT3QOP3.1 size=202361 2024-05-09T20:16:49.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.216+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=M6QAJFT3QOP3 tag=KSSLP9TDE5UZ type=write size=2299513 2024-05-09T20:16:49.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.224+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~2UQ7EREIPM20.0 size=707875 2024-05-09T20:16:49.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.224+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=2UQ7EREIPM20 tag=SHQEBJ5TR89Y type=write size=1756451 2024-05-09T20:16:49.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.228+0000 7fdc882e58c0 1 < completed write operation key=NBAECV9L85UK tag=ZPEBHOZ7B81M type=write size=233146 2024-05-09T20:16:49.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.232+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~J6VZTELHDJCE.0 size=748470 2024-05-09T20:16:49.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.232+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=J6VZTELHDJCE tag=2WIR2QZ56IES type=write size=1797046 2024-05-09T20:16:49.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.236+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=HLGJU2E2DUCI type=del 2024-05-09T20:16:49.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.240+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~2CIMNIXTB1MZ.0 size=1048576 2024-05-09T20:16:49.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.244+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~2CIMNIXTB1MZ.1 size=1048576 2024-05-09T20:16:49.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.244+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~2CIMNIXTB1MZ.2 size=986693 2024-05-09T20:16:49.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.244+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=2CIMNIXTB1MZ tag=L7D2JU5ROLID type=write size=4132421 2024-05-09T20:16:49.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.248+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~8HPSZ9O260OY.0 size=1048576 2024-05-09T20:16:49.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.252+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~8HPSZ9O260OY.1 size=1048576 2024-05-09T20:16:49.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.252+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~8HPSZ9O260OY.2 size=1007326 2024-05-09T20:16:49.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.256+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=8HPSZ9O260OY tag=H2DDDSMGUT5I type=write size=4153054 2024-05-09T20:16:49.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.264+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~DPVAZHR4BBCV.0 size=1048576 2024-05-09T20:16:49.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.264+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~DPVAZHR4BBCV.1 size=1048576 2024-05-09T20:16:49.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.268+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~DPVAZHR4BBCV.2 size=1043754 2024-05-09T20:16:49.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.268+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=DPVAZHR4BBCV tag=JRF4A7R7WS22 type=write size=4189482 2024-05-09T20:16:49.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.272+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=HITPQU9BY627 type=del 2024-05-09T20:16:49.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.276+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=BXDI5RJNPD04 type=write size=527827 2024-05-09T20:16:49.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.276+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=OX3ZKNRJDM2S type=del 2024-05-09T20:16:49.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.284+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~AKRW7V4NRPKJ.0 size=1048576 2024-05-09T20:16:49.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.288+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~AKRW7V4NRPKJ.1 size=368559 2024-05-09T20:16:49.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.288+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=AKRW7V4NRPKJ tag=QDC84RYVHKU8 type=write size=2465711 2024-05-09T20:16:49.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.288+0000 7fdc882e58c0 1 < canceled operation key=CNV7NHP27JZ7 tag=G3U4S9DDZVHW type=del size=3610094 2024-05-09T20:16:49.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.296+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~JYN0F01CFRUU.0 size=1048576 2024-05-09T20:16:49.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.300+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~JYN0F01CFRUU.1 size=859000 2024-05-09T20:16:49.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.300+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=JYN0F01CFRUU tag=WR5XY1ZR4VLI type=write size=2956152 2024-05-09T20:16:49.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.304+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=3G5MN1IMIQI9 type=del 2024-05-09T20:16:49.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.304+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=2LKPMYT5IYKY type=del size=3208699 2024-05-09T20:16:49.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.308+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=34NAKQAP3BYG type=del 2024-05-09T20:16:49.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.312+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=W8CG42PT1586 type=del size=2976575 2024-05-09T20:16:49.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.320+0000 7fdc882e58c0 1 < suggested update operation key=BS0T1OQOBS67 tag=WBR118HFFUHZ type=del size=2589783 2024-05-09T20:16:49.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.320+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=SUPBVGS9GJ0W type=del size=2847610 2024-05-09T20:16:49.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.324+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=08HFZYJLY4PW type=del 2024-05-09T20:16:49.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.328+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=NAC1T2JBFJUE type=del size=1899904 2024-05-09T20:16:49.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.336+0000 7fdc882e58c0 1 < suggested remove operation key=O07UHAIJZF0R tag=IMT4S7OX1DVY type=del 2024-05-09T20:16:49.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.340+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=RYRQ7Y0PWU20 type=del size=1709365 2024-05-09T20:16:49.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.344+0000 7fdc882e58c0 1 < suggested remove operation key=T6T3IOW9VPA0 tag=8E4GPEG9H7L2 type=del 2024-05-09T20:16:49.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.348+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=EHAXXUE2M0IJ type=del size=3993826 2024-05-09T20:16:49.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.352+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=HVSI3WFGFFXR type=del 2024-05-09T20:16:49.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.360+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~4SM0FS82PZ0L.0 size=1048576 2024-05-09T20:16:49.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.360+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~4SM0FS82PZ0L.1 size=1048576 2024-05-09T20:16:49.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.364+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~4SM0FS82PZ0L.2 size=62286 2024-05-09T20:16:49.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.364+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=4SM0FS82PZ0L tag=IICLK56D7H93 type=write size=3208014 2024-05-09T20:16:49.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.368+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=YF70NG4DE4HS type=write size=265701 2024-05-09T20:16:49.364 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.372+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=6YUXVQBL4I3B type=del size=3807231 2024-05-09T20:16:49.371 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.380+0000 7fdc882e58c0 1 < suggested update operation key=BS0T1OQOBS67 tag=AXFCI0KSP7E0 type=write size=227099 2024-05-09T20:16:49.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.384+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=T1061V5YUFDJ type=del size=2177409 2024-05-09T20:16:49.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.384+0000 7fdc882e58c0 1 < canceled part key=_multipart_CNV7NHP27JZ7.2~2UQ7EREIPM20.0 size=707875 2024-05-09T20:16:49.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.388+0000 7fdc882e58c0 1 < canceled multipart upload key=CNV7NHP27JZ7 upload=2UQ7EREIPM20 tag=SHQEBJ5TR89Y type=write size=1756451 2024-05-09T20:16:49.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.396+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~MAVJ6PLIRG9F.0 size=170117 2024-05-09T20:16:49.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.396+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=MAVJ6PLIRG9F tag=BCICAEZ21AZ9 type=write size=1218693 2024-05-09T20:16:49.390 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.400+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~J6VZTELHDJCE.0 size=748470 2024-05-09T20:16:49.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.404+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=J6VZTELHDJCE tag=2WIR2QZ56IES type=write size=1797046 2024-05-09T20:16:49.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.412+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~GKOJES1FXONB.0 size=440592 2024-05-09T20:16:49.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.412+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=GKOJES1FXONB tag=1SBOG02KGC4F type=write size=1489168 2024-05-09T20:16:49.406 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.412+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~2CIMNIXTB1MZ.0 size=1048576 2024-05-09T20:16:49.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.416+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~2CIMNIXTB1MZ.1 size=1048576 2024-05-09T20:16:49.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.420+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~2CIMNIXTB1MZ.2 size=986693 2024-05-09T20:16:49.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.424+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=2CIMNIXTB1MZ tag=L7D2JU5ROLID type=write size=4132421 2024-05-09T20:16:49.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.432+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~2VZT833MFOF0.0 size=779137 2024-05-09T20:16:49.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.432+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=2VZT833MFOF0 tag=FTU4R2SY5AI2 type=write size=1827713 2024-05-09T20:16:49.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.436+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~DPVAZHR4BBCV.0 size=1048576 2024-05-09T20:16:49.430 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.440+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~DPVAZHR4BBCV.1 size=1048576 2024-05-09T20:16:49.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.440+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~DPVAZHR4BBCV.2 size=1043754 2024-05-09T20:16:49.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.448+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=DPVAZHR4BBCV tag=JRF4A7R7WS22 type=write size=4189482 2024-05-09T20:16:49.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.452+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~QPJ3B0UYION9.0 size=1048576 2024-05-09T20:16:49.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.456+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~QPJ3B0UYION9.1 size=119143 2024-05-09T20:16:49.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.456+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=QPJ3B0UYION9 tag=KXPO1D3GAZVL type=write size=2216295 2024-05-09T20:16:49.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.460+0000 7fdc882e58c0 1 < completed write operation key=CNV7NHP27JZ7 tag=BXDI5RJNPD04 type=write size=527827 2024-05-09T20:16:49.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.464+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=V3WUFNLZJN0P type=write size=472055 2024-05-09T20:16:49.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.468+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~AKRW7V4NRPKJ.0 size=1048576 2024-05-09T20:16:49.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.472+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~AKRW7V4NRPKJ.1 size=368559 2024-05-09T20:16:49.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.476+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=AKRW7V4NRPKJ tag=QDC84RYVHKU8 type=write size=2465711 2024-05-09T20:16:49.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.480+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=6D1FWYXMUB4C type=del size=3616311 2024-05-09T20:16:49.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.484+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~JYN0F01CFRUU.0 size=1048576 2024-05-09T20:16:49.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.488+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~JYN0F01CFRUU.1 size=859000 2024-05-09T20:16:49.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.492+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=JYN0F01CFRUU tag=WR5XY1ZR4VLI type=write size=2956152 2024-05-09T20:16:49.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.496+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=IVRJJAOC9KV0 type=del size=2777280 2024-05-09T20:16:49.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.504+0000 7fdc882e58c0 1 < suggested update operation key=NYHR6GTPQYZS tag=2LKPMYT5IYKY type=del size=3208699 2024-05-09T20:16:49.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.508+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~TKSVFTQ1JVEU.0 size=633728 2024-05-09T20:16:49.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.508+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=TKSVFTQ1JVEU tag=L06RE1RMFNZJ type=write size=1682304 2024-05-09T20:16:49.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.516+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=W8CG42PT1586 type=del 2024-05-09T20:16:49.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.520+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=I9NCNQWO1JWU type=del size=2849328 2024-05-09T20:16:49.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.524+0000 7fdc882e58c0 1 < suggested remove operation key=CNV7NHP27JZ7 tag=SUPBVGS9GJ0W type=del 2024-05-09T20:16:49.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.528+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=X6Q3HDRUG05F type=del size=2234431 2024-05-09T20:16:49.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.536+0000 7fdc882e58c0 1 < suggested update operation key=IZPPFLWVPRKO tag=NAC1T2JBFJUE type=del size=1899904 2024-05-09T20:16:49.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.540+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~O2DVP3L1WUYZ.0 size=756337 2024-05-09T20:16:49.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.540+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=O2DVP3L1WUYZ tag=VMUA6SNOV9K5 type=write size=1804913 2024-05-09T20:16:49.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.544+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=RYRQ7Y0PWU20 type=del 2024-05-09T20:16:49.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.548+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=Y7VBK008465Q type=del size=1777173 2024-05-09T20:16:49.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.552+0000 7fdc882e58c0 -1 < failed to complete operation key=NYHR6GTPQYZS tag=EHAXXUE2M0IJ type=del size=3993826: (22) Invalid argument 2024-05-09T20:16:49.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.556+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=KVFEQB7AZ5OS type=del size=969261 2024-05-09T20:16:49.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.560+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~4SM0FS82PZ0L.0 size=1048576 2024-05-09T20:16:49.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.564+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~4SM0FS82PZ0L.1 size=1048576 2024-05-09T20:16:49.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.564+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~4SM0FS82PZ0L.2 size=62286 2024-05-09T20:16:49.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.572+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=4SM0FS82PZ0L tag=IICLK56D7H93 type=write size=3208014 2024-05-09T20:16:49.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.576+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=2H62KOA0QTU0 type=del size=2736849 2024-05-09T20:16:49.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.576+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=6YUXVQBL4I3B type=del 2024-05-09T20:16:49.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.580+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=5CY0YJ6FSYUB type=del size=428964 2024-05-09T20:16:49.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.584+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=T1061V5YUFDJ type=del 2024-05-09T20:16:49.582 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.592+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~IL6GNY19FPH7.0 size=1048576 2024-05-09T20:16:49.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.592+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~IL6GNY19FPH7.1 size=245092 2024-05-09T20:16:49.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.592+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=IL6GNY19FPH7 tag=8WJZNUCA0CDD type=write size=2342244 2024-05-09T20:16:49.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.596+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~MAVJ6PLIRG9F.0 size=170117 2024-05-09T20:16:49.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.604+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=MAVJ6PLIRG9F tag=BCICAEZ21AZ9 type=write size=1218693 2024-05-09T20:16:49.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.608+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~NRBOGWOQPMN8.0 size=1048576 2024-05-09T20:16:49.603 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~NRBOGWOQPMN8.1 size=1048576 2024-05-09T20:16:49.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.616+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~NRBOGWOQPMN8.2 size=113132 2024-05-09T20:16:49.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.616+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=NRBOGWOQPMN8 tag=JQ1LTLHG3EHU type=write size=3258860 2024-05-09T20:16:49.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.616+0000 7fdc882e58c0 1 < canceled part key=_multipart_54R9XPO62J0M.2~GKOJES1FXONB.0 size=440592 2024-05-09T20:16:49.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.620+0000 7fdc882e58c0 1 < canceled multipart upload key=54R9XPO62J0M upload=GKOJES1FXONB tag=1SBOG02KGC4F type=write size=1489168 2024-05-09T20:16:49.616 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.624+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=ZJ79WZRGFV0O type=del size=896858 2024-05-09T20:16:49.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.628+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~2VZT833MFOF0.0 size=779137 2024-05-09T20:16:49.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.632+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=2VZT833MFOF0 tag=FTU4R2SY5AI2 type=write size=1827713 2024-05-09T20:16:49.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.636+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=1XEBG8MUF0KI type=del size=2797621 2024-05-09T20:16:49.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.640+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~QPJ3B0UYION9.0 size=1048576 2024-05-09T20:16:49.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.644+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~QPJ3B0UYION9.1 size=119143 2024-05-09T20:16:49.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.648+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=QPJ3B0UYION9 tag=KXPO1D3GAZVL type=write size=2216295 2024-05-09T20:16:49.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.656+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~N3LIGNA02T73.0 size=1048576 2024-05-09T20:16:49.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.660+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~N3LIGNA02T73.1 size=1048576 2024-05-09T20:16:49.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.660+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~N3LIGNA02T73.2 size=693917 2024-05-09T20:16:49.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.660+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=N3LIGNA02T73 tag=2DK4UNCOYNWX type=write size=3839645 2024-05-09T20:16:49.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.664+0000 7fdc882e58c0 1 < completed write operation key=OGVA2778MJWU tag=V3WUFNLZJN0P type=write size=472055 2024-05-09T20:16:49.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.668+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=EC5S9QRFYXIZ type=write size=72224 2024-05-09T20:16:49.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.672+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=6D1FWYXMUB4C type=del 2024-05-09T20:16:49.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.672+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=SD2SZ1AJV3JE type=del size=292582 2024-05-09T20:16:49.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.680+0000 7fdc882e58c0 1 < suggested update operation key=HPK2T0629B1P tag=IVRJJAOC9KV0 type=del size=2777280 2024-05-09T20:16:49.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.684+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=PQSUHH1S0YEB type=del size=3899171 2024-05-09T20:16:49.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.688+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~TKSVFTQ1JVEU.0 size=633728 2024-05-09T20:16:49.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.696+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=TKSVFTQ1JVEU tag=L06RE1RMFNZJ type=write size=1682304 2024-05-09T20:16:49.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.700+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=JJ5BHRI5PA85 type=del size=2902317 2024-05-09T20:16:49.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.708+0000 7fdc882e58c0 1 < suggested update operation key=OGVA2778MJWU tag=I9NCNQWO1JWU type=del size=2849328 2024-05-09T20:16:49.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.712+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=PX4S3VVQPWXX type=del size=2927189 2024-05-09T20:16:49.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.716+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=X6Q3HDRUG05F type=del 2024-05-09T20:16:49.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.720+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~Q04CA0RK5WL5.0 size=1048576 2024-05-09T20:16:49.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.724+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~Q04CA0RK5WL5.1 size=240643 2024-05-09T20:16:49.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.724+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=Q04CA0RK5WL5 tag=2M3UNMP0WVFB type=write size=2337795 2024-05-09T20:16:49.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.728+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~O2DVP3L1WUYZ.0 size=756337 2024-05-09T20:16:49.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.736+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=O2DVP3L1WUYZ tag=VMUA6SNOV9K5 type=write size=1804913 2024-05-09T20:16:49.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.740+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~ECDLXK4U5P0M.0 size=31820 2024-05-09T20:16:49.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.740+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=ECDLXK4U5P0M tag=3BVT7DSKVNRP type=write size=1080396 2024-05-09T20:16:49.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.744+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=Y7VBK008465Q type=del 2024-05-09T20:16:49.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.748+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=PW4JIZ7R8Q4N type=del size=782724 2024-05-09T20:16:49.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.752+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=KVFEQB7AZ5OS type=del 2024-05-09T20:16:49.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.756+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=D5F83K6GN48U type=del size=4019178 2024-05-09T20:16:49.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.760+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=2H62KOA0QTU0 type=del 2024-05-09T20:16:49.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.764+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~Z7AERB9DY13V.0 size=1048576 2024-05-09T20:16:49.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.768+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~Z7AERB9DY13V.1 size=1048576 2024-05-09T20:16:49.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.772+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~Z7AERB9DY13V.2 size=503517 2024-05-09T20:16:49.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.772+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=Z7AERB9DY13V tag=XKVA1IEO14EN type=write size=3649245 2024-05-09T20:16:49.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.772+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=5CY0YJ6FSYUB type=del 2024-05-09T20:16:49.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~HX5HDFBOEJ1H.0 size=1048576 2024-05-09T20:16:49.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~HX5HDFBOEJ1H.1 size=326107 2024-05-09T20:16:49.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.784+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=HX5HDFBOEJ1H tag=S624OQAAMM4A type=write size=2423259 2024-05-09T20:16:49.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.788+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~IL6GNY19FPH7.0 size=1048576 2024-05-09T20:16:49.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.788+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~IL6GNY19FPH7.1 size=245092 2024-05-09T20:16:49.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.796+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=IL6GNY19FPH7 tag=8WJZNUCA0CDD type=write size=2342244 2024-05-09T20:16:49.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.804+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~NGN5RURRDCC2.0 size=690816 2024-05-09T20:16:49.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.804+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=NGN5RURRDCC2 tag=Y5Y7T9TBTQH2 type=write size=1739392 2024-05-09T20:16:49.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.808+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~NRBOGWOQPMN8.0 size=1048576 2024-05-09T20:16:49.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.808+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~NRBOGWOQPMN8.1 size=1048576 2024-05-09T20:16:49.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.812+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~NRBOGWOQPMN8.2 size=113132 2024-05-09T20:16:49.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.816+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=NRBOGWOQPMN8 tag=JQ1LTLHG3EHU type=write size=3258860 2024-05-09T20:16:49.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.820+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=VPR3JPFHAKWR type=del size=2103975 2024-05-09T20:16:49.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.824+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=ZJ79WZRGFV0O type=del 2024-05-09T20:16:49.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.828+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=BUSFLKCZLM52 type=del size=3130710 2024-05-09T20:16:49.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.832+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=1XEBG8MUF0KI type=del 2024-05-09T20:16:49.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.832+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=BTPS8PHU6SYN type=del size=2379889 2024-05-09T20:16:49.828 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.836+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~N3LIGNA02T73.0 size=1048576 2024-05-09T20:16:49.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.840+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~N3LIGNA02T73.1 size=1048576 2024-05-09T20:16:49.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.844+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~N3LIGNA02T73.2 size=693917 2024-05-09T20:16:49.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.848+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=N3LIGNA02T73 tag=2DK4UNCOYNWX type=write size=3839645 2024-05-09T20:16:49.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.852+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=MGIW9BVGC9R3 type=del size=1675770 2024-05-09T20:16:49.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.852+0000 7fdc882e58c0 1 < completed write operation key=NNGWUSCETMH4 tag=EC5S9QRFYXIZ type=write size=72224 2024-05-09T20:16:49.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.856+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=94DRD2L2JSUW type=del size=3757809 2024-05-09T20:16:49.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.860+0000 7fdc882e58c0 -1 < failed to complete operation key=OGVA2778MJWU tag=SD2SZ1AJV3JE type=del size=292582: (22) Invalid argument 2024-05-09T20:16:49.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.864+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~QYKXIL8QCAUN.0 size=1048576 2024-05-09T20:16:49.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.868+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~QYKXIL8QCAUN.1 size=1048576 2024-05-09T20:16:49.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.868+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~QYKXIL8QCAUN.2 size=875255 2024-05-09T20:16:49.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.868+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=QYKXIL8QCAUN tag=FY5A2XAAN97D type=write size=4020983 2024-05-09T20:16:49.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.872+0000 7fdc882e58c0 1 < canceled operation key=IZPPFLWVPRKO tag=PQSUHH1S0YEB type=del size=3899171 2024-05-09T20:16:49.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.876+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~MC3TBN8A18EX.0 size=903017 2024-05-09T20:16:49.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.876+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=MC3TBN8A18EX tag=UEYF4WE65SUH type=write size=1951593 2024-05-09T20:16:49.872 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.880+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=JJ5BHRI5PA85 type=del 2024-05-09T20:16:49.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.884+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~Z02JEP6NAPXM.0 size=169692 2024-05-09T20:16:49.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.884+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=Z02JEP6NAPXM tag=0H5QTYH5I2HK type=write size=1218268 2024-05-09T20:16:49.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.892+0000 7fdc882e58c0 1 < suggested update operation key=VZ7XOAT899GU tag=PX4S3VVQPWXX type=del size=2927189 2024-05-09T20:16:49.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.896+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~H0P13B1DXTG8.0 size=1048576 2024-05-09T20:16:49.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~H0P13B1DXTG8.1 size=1048576 2024-05-09T20:16:49.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~H0P13B1DXTG8.2 size=316060 2024-05-09T20:16:49.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.900+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=H0P13B1DXTG8 tag=SVQ3R91QAJ6F type=write size=3461788 2024-05-09T20:16:49.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.904+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~Q04CA0RK5WL5.0 size=1048576 2024-05-09T20:16:49.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.908+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~Q04CA0RK5WL5.1 size=240643 2024-05-09T20:16:49.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.912+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=Q04CA0RK5WL5 tag=2M3UNMP0WVFB type=write size=2337795 2024-05-09T20:16:49.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.916+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~51G53CI35DXR.0 size=1048576 2024-05-09T20:16:49.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.920+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~51G53CI35DXR.1 size=1048576 2024-05-09T20:16:49.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.920+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~51G53CI35DXR.2 size=576434 2024-05-09T20:16:49.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.920+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=51G53CI35DXR tag=EQFM10YPEONB type=write size=3722162 2024-05-09T20:16:49.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.924+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~ECDLXK4U5P0M.0 size=31820 2024-05-09T20:16:49.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.928+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=ECDLXK4U5P0M tag=3BVT7DSKVNRP type=write size=1080396 2024-05-09T20:16:49.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.932+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=7VNXFHRY2VTU type=del size=587862 2024-05-09T20:16:49.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.936+0000 7fdc882e58c0 1 < canceled operation key=5H2GWM9LQELZ tag=PW4JIZ7R8Q4N type=del size=782724 2024-05-09T20:16:49.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.940+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=S8CBUNGL6BDF type=del size=266564 2024-05-09T20:16:49.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.940+0000 7fdc882e58c0 1 < canceled operation key=OGVA2778MJWU tag=D5F83K6GN48U type=del size=4019178 2024-05-09T20:16:49.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.948+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~NW44BZF5AKCE.0 size=1048576 2024-05-09T20:16:49.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.948+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~NW44BZF5AKCE.1 size=1048576 2024-05-09T20:16:49.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.952+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~NW44BZF5AKCE.2 size=936791 2024-05-09T20:16:49.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.952+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=NW44BZF5AKCE tag=WZ818Q3G2OZC type=write size=4082519 2024-05-09T20:16:49.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.956+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~Z7AERB9DY13V.0 size=1048576 2024-05-09T20:16:49.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.960+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~Z7AERB9DY13V.1 size=1048576 2024-05-09T20:16:49.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.960+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~Z7AERB9DY13V.2 size=503517 2024-05-09T20:16:49.958 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.964+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=Z7AERB9DY13V tag=XKVA1IEO14EN type=write size=3649245 2024-05-09T20:16:49.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.972+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~A3RPXLRD9807.0 size=1048576 2024-05-09T20:16:49.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.972+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~A3RPXLRD9807.1 size=1048576 2024-05-09T20:16:49.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.976+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~A3RPXLRD9807.2 size=290480 2024-05-09T20:16:49.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.976+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=A3RPXLRD9807 tag=VPZ9JOOE29E7 type=write size=3436208 2024-05-09T20:16:49.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.980+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~HX5HDFBOEJ1H.0 size=1048576 2024-05-09T20:16:49.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.980+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~HX5HDFBOEJ1H.1 size=326107 2024-05-09T20:16:49.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.988+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=HX5HDFBOEJ1H tag=S624OQAAMM4A type=write size=2423259 2024-05-09T20:16:49.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.992+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~ZNYF31EVXUBN.0 size=1048576 2024-05-09T20:16:49.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.996+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~ZNYF31EVXUBN.1 size=366875 2024-05-09T20:16:49.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.996+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=ZNYF31EVXUBN tag=2SC3Z5YCL9E0 type=write size=2464027 2024-05-09T20:16:49.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:49.996+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~NGN5RURRDCC2.0 size=690816 2024-05-09T20:16:49.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.004+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=NGN5RURRDCC2 tag=Y5Y7T9TBTQH2 type=write size=1739392 2024-05-09T20:16:49.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.008+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~OZBK1TAODUNU.0 size=1048576 2024-05-09T20:16:50.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.008+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~OZBK1TAODUNU.1 size=1048576 2024-05-09T20:16:50.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.012+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~OZBK1TAODUNU.2 size=302542 2024-05-09T20:16:50.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.012+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=OZBK1TAODUNU tag=VUVFKGNNQINF type=write size=3448270 2024-05-09T20:16:50.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.016+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=VPR3JPFHAKWR type=del 2024-05-09T20:16:50.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.016+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=CTPADQP1YPVG type=write size=13789 2024-05-09T20:16:50.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.024+0000 7fdc882e58c0 1 < suggested update operation key=NYHR6GTPQYZS tag=BUSFLKCZLM52 type=del size=3130710 2024-05-09T20:16:50.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.028+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=SY403TRCP2BA type=write size=705748 2024-05-09T20:16:50.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.028+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=BTPS8PHU6SYN type=del 2024-05-09T20:16:50.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.036+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~PK1FSJH4O5KH.0 size=1048576 2024-05-09T20:16:50.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.036+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~PK1FSJH4O5KH.1 size=663435 2024-05-09T20:16:50.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.036+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=PK1FSJH4O5KH tag=Y9X2V8VNHGTK type=write size=2760587 2024-05-09T20:16:50.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.040+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=MGIW9BVGC9R3 type=del 2024-05-09T20:16:50.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.040+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=NJ5C61NUAOJP type=del size=286834 2024-05-09T20:16:50.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.048+0000 7fdc882e58c0 1 < suggested update operation key=O07UHAIJZF0R tag=94DRD2L2JSUW type=del size=3757809 2024-05-09T20:16:50.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.048+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=JVZ0BQM49XBR type=del size=157354 2024-05-09T20:16:50.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.052+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~QYKXIL8QCAUN.0 size=1048576 2024-05-09T20:16:50.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.056+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~QYKXIL8QCAUN.1 size=1048576 2024-05-09T20:16:50.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.056+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~QYKXIL8QCAUN.2 size=875255 2024-05-09T20:16:50.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.064+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=QYKXIL8QCAUN tag=FY5A2XAAN97D type=write size=4020983 2024-05-09T20:16:50.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.068+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=04SOJ931B87C type=del size=2183859 2024-05-09T20:16:50.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.068+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~MC3TBN8A18EX.0 size=903017 2024-05-09T20:16:50.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.076+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=MC3TBN8A18EX tag=UEYF4WE65SUH type=write size=1951593 2024-05-09T20:16:50.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.080+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~SU6L301T7AJO.0 size=1048576 2024-05-09T20:16:50.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.084+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~SU6L301T7AJO.1 size=414827 2024-05-09T20:16:50.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.084+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=SU6L301T7AJO tag=YNG3DQXIWUQ3 type=write size=2511979 2024-05-09T20:16:50.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.084+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~Z02JEP6NAPXM.0 size=169692 2024-05-09T20:16:50.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.088+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=Z02JEP6NAPXM tag=0H5QTYH5I2HK type=write size=1218268 2024-05-09T20:16:50.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.092+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=FO8K3HWBH4Q4 type=write size=619897 2024-05-09T20:16:50.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.096+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~H0P13B1DXTG8.0 size=1048576 2024-05-09T20:16:50.088 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.096+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~H0P13B1DXTG8.1 size=1048576 2024-05-09T20:16:50.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.100+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~H0P13B1DXTG8.2 size=316060 2024-05-09T20:16:50.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.104+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=H0P13B1DXTG8 tag=SVQ3R91QAJ6F type=write size=3461788 2024-05-09T20:16:50.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.108+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=3R2QPLA3FY9B type=del size=2559583 2024-05-09T20:16:50.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.112+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~51G53CI35DXR.0 size=1048576 2024-05-09T20:16:50.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.116+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~51G53CI35DXR.1 size=1048576 2024-05-09T20:16:50.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.120+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~51G53CI35DXR.2 size=576434 2024-05-09T20:16:50.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.124+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=51G53CI35DXR tag=EQFM10YPEONB type=write size=3722162 2024-05-09T20:16:50.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.128+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=F20PK1JH46N2 type=del size=157466 2024-05-09T20:16:50.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.132+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=7VNXFHRY2VTU type=del 2024-05-09T20:16:50.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.136+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~6CDUUEIYQ22U.0 size=1048576 2024-05-09T20:16:50.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.140+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~6CDUUEIYQ22U.1 size=1048576 2024-05-09T20:16:50.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.140+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~6CDUUEIYQ22U.2 size=612286 2024-05-09T20:16:50.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.140+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=6CDUUEIYQ22U tag=LV3OIMV24W1S type=write size=3758014 2024-05-09T20:16:50.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.144+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=S8CBUNGL6BDF type=del 2024-05-09T20:16:50.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.148+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=104B5D33NHJK type=del size=3138437 2024-05-09T20:16:50.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.152+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~NW44BZF5AKCE.0 size=1048576 2024-05-09T20:16:50.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.152+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~NW44BZF5AKCE.1 size=1048576 2024-05-09T20:16:50.149 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.156+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~NW44BZF5AKCE.2 size=936791 2024-05-09T20:16:50.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.164+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=NW44BZF5AKCE tag=WZ818Q3G2OZC type=write size=4082519 2024-05-09T20:16:50.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.164+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=Z6FM32VSO5WL type=del size=1673620 2024-05-09T20:16:50.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.168+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~A3RPXLRD9807.0 size=1048576 2024-05-09T20:16:50.164 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.172+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~A3RPXLRD9807.1 size=1048576 2024-05-09T20:16:50.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.176+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~A3RPXLRD9807.2 size=290480 2024-05-09T20:16:50.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.180+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=A3RPXLRD9807 tag=VPZ9JOOE29E7 type=write size=3436208 2024-05-09T20:16:50.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.184+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~WGWFCN03J4G1.0 size=585080 2024-05-09T20:16:50.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.184+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=WGWFCN03J4G1 tag=FM7C3UXJTS68 type=write size=1633656 2024-05-09T20:16:50.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.188+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~ZNYF31EVXUBN.0 size=1048576 2024-05-09T20:16:50.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.192+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~ZNYF31EVXUBN.1 size=366875 2024-05-09T20:16:50.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.196+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=ZNYF31EVXUBN tag=2SC3Z5YCL9E0 type=write size=2464027 2024-05-09T20:16:50.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.200+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=0OSK17OJAF62 type=del size=1917205 2024-05-09T20:16:50.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.204+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~OZBK1TAODUNU.0 size=1048576 2024-05-09T20:16:50.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.204+0000 7fdc882e58c0 1 < canceled part key=_multipart_IZPPFLWVPRKO.2~OZBK1TAODUNU.1 size=1048576 2024-05-09T20:16:50.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.204+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~OZBK1TAODUNU.2 size=302542 2024-05-09T20:16:50.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.212+0000 7fdc882e58c0 1 < canceled multipart upload key=IZPPFLWVPRKO upload=OZBK1TAODUNU tag=VUVFKGNNQINF type=write size=3448270 2024-05-09T20:16:50.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.212+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=C17G63D9HGX7 type=del size=3354188 2024-05-09T20:16:50.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.216+0000 7fdc882e58c0 1 < completed write operation key=CNV7NHP27JZ7 tag=CTPADQP1YPVG type=write size=13789 2024-05-09T20:16:50.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.220+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=B9B3GGPK11RL type=del size=3405506 2024-05-09T20:16:50.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.224+0000 7fdc882e58c0 1 < completed write operation key=EK78CNKUHNKO tag=SY403TRCP2BA type=write size=705748 2024-05-09T20:16:50.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.228+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=02XQA86JQ8SC type=del size=3961579 2024-05-09T20:16:50.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.232+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~PK1FSJH4O5KH.0 size=1048576 2024-05-09T20:16:50.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.232+0000 7fdc882e58c0 1 < canceled part key=_multipart_WC5YH0OU51WX.2~PK1FSJH4O5KH.1 size=663435 2024-05-09T20:16:50.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.236+0000 7fdc882e58c0 1 < canceled multipart upload key=WC5YH0OU51WX upload=PK1FSJH4O5KH tag=Y9X2V8VNHGTK type=write size=2760587 2024-05-09T20:16:50.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.244+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~YS0VZ7AJ1UHY.0 size=1048576 2024-05-09T20:16:50.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.244+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~YS0VZ7AJ1UHY.1 size=1048576 2024-05-09T20:16:50.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.248+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~YS0VZ7AJ1UHY.2 size=1036458 2024-05-09T20:16:50.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.248+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=YS0VZ7AJ1UHY tag=NI6CSMZ2HO70 type=write size=4182186 2024-05-09T20:16:50.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.252+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=NJ5C61NUAOJP type=del 2024-05-09T20:16:50.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.252+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=D8DNNXTM3HA7 type=write size=145238 2024-05-09T20:16:50.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.256+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=JVZ0BQM49XBR type=del 2024-05-09T20:16:50.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.264+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~JDM4OQKECA1S.0 size=1048576 2024-05-09T20:16:50.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.268+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~JDM4OQKECA1S.1 size=921836 2024-05-09T20:16:50.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.268+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=JDM4OQKECA1S tag=33EGJAWA0RWH type=write size=3018988 2024-05-09T20:16:50.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.272+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=04SOJ931B87C type=del 2024-05-09T20:16:50.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.272+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=9WM2Q05V1GAV type=del size=1806030 2024-05-09T20:16:50.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.276+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~SU6L301T7AJO.0 size=1048576 2024-05-09T20:16:50.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.280+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~SU6L301T7AJO.1 size=414827 2024-05-09T20:16:50.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.284+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=SU6L301T7AJO tag=YNG3DQXIWUQ3 type=write size=2511979 2024-05-09T20:16:50.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.292+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~8ME0VILLXX4A.0 size=1048576 2024-05-09T20:16:50.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.296+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~8ME0VILLXX4A.1 size=1048576 2024-05-09T20:16:50.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.296+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~8ME0VILLXX4A.2 size=909472 2024-05-09T20:16:50.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.296+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=8ME0VILLXX4A tag=6B35HIESB5FY type=write size=4055200 2024-05-09T20:16:50.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.300+0000 7fdc882e58c0 1 < completed write operation key=QM53IENH9CX4 tag=FO8K3HWBH4Q4 type=write size=619897 2024-05-09T20:16:50.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.304+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=KDJYYL8Y41GE type=del size=2483521 2024-05-09T20:16:50.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.312+0000 7fdc882e58c0 1 < suggested remove operation key=5H2GWM9LQELZ tag=3R2QPLA3FY9B type=del 2024-05-09T20:16:50.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.316+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=JY493HNDYJBI type=del size=842401 2024-05-09T20:16:50.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.320+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=F20PK1JH46N2 type=del 2024-05-09T20:16:50.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.324+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=T7GF3NPPD5LH type=del size=438726 2024-05-09T20:16:50.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.328+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~6CDUUEIYQ22U.0 size=1048576 2024-05-09T20:16:50.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.332+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~6CDUUEIYQ22U.1 size=1048576 2024-05-09T20:16:50.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.336+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~6CDUUEIYQ22U.2 size=612286 2024-05-09T20:16:50.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.340+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=6CDUUEIYQ22U tag=LV3OIMV24W1S type=write size=3758014 2024-05-09T20:16:50.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.344+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=BS80R9OTICFH type=del size=1546152 2024-05-09T20:16:50.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.352+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=104B5D33NHJK type=del 2024-05-09T20:16:50.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.356+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=4G5OHXSPRCMZ type=del size=3257143 2024-05-09T20:16:50.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.360+0000 7fdc882e58c0 1 < suggested remove operation key=SPXHCWZICCWY tag=Z6FM32VSO5WL type=del 2024-05-09T20:16:50.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.368+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~LDTCM3HINYUB.0 size=917695 2024-05-09T20:16:50.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.368+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=LDTCM3HINYUB tag=S4PXJD7E16VH type=write size=1966271 2024-05-09T20:16:50.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.372+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~WGWFCN03J4G1.0 size=585080 2024-05-09T20:16:50.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.376+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=WGWFCN03J4G1 tag=FM7C3UXJTS68 type=write size=1633656 2024-05-09T20:16:50.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.380+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=R0R2359Z8UDY type=del size=988011 2024-05-09T20:16:50.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.384+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=0OSK17OJAF62 type=del 2024-05-09T20:16:50.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.388+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=A74MUWV4SBLA type=del size=1305963 2024-05-09T20:16:50.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.392+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=C17G63D9HGX7 type=del 2024-05-09T20:16:50.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.396+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=CAKCMX71Z8EE type=del size=2975509 2024-05-09T20:16:50.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.396+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=B9B3GGPK11RL type=del 2024-05-09T20:16:50.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.404+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~6U1ZIIQGYSV6.0 size=829681 2024-05-09T20:16:50.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.404+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=6U1ZIIQGYSV6 tag=B6RRR9GGZVC4 type=write size=1878257 2024-05-09T20:16:50.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.408+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=02XQA86JQ8SC type=del 2024-05-09T20:16:50.405 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.412+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~0BBCSZ06WUB9.0 size=1048576 2024-05-09T20:16:50.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.416+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~0BBCSZ06WUB9.1 size=821210 2024-05-09T20:16:50.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.416+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=0BBCSZ06WUB9 tag=IEBKCMB8IRFJ type=write size=2918362 2024-05-09T20:16:50.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.420+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~YS0VZ7AJ1UHY.0 size=1048576 2024-05-09T20:16:50.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.424+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~YS0VZ7AJ1UHY.1 size=1048576 2024-05-09T20:16:50.419 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.428+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~YS0VZ7AJ1UHY.2 size=1036458 2024-05-09T20:16:50.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.432+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=YS0VZ7AJ1UHY tag=NI6CSMZ2HO70 type=write size=4182186 2024-05-09T20:16:50.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.440+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~PAKR9N63VG9K.0 size=255297 2024-05-09T20:16:50.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.440+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=PAKR9N63VG9K tag=B6LIEQT32EJU type=write size=1303873 2024-05-09T20:16:50.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.444+0000 7fdc882e58c0 1 < completed write operation key=J2Q86HKRMAKE tag=D8DNNXTM3HA7 type=write size=145238 2024-05-09T20:16:50.440 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.448+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~6AGJ9TPX3US5.0 size=180692 2024-05-09T20:16:50.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.448+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=6AGJ9TPX3US5 tag=M3VFO6UBNNHW type=write size=1229268 2024-05-09T20:16:50.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.452+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~JDM4OQKECA1S.0 size=1048576 2024-05-09T20:16:50.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.456+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~JDM4OQKECA1S.1 size=921836 2024-05-09T20:16:50.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.460+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=JDM4OQKECA1S tag=33EGJAWA0RWH type=write size=3018988 2024-05-09T20:16:50.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.464+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=9UIGU3ZLIR56 type=del size=463330 2024-05-09T20:16:50.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.468+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=9WM2Q05V1GAV type=del 2024-05-09T20:16:50.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.472+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=OYIQFUN8MVLT type=del size=1952608 2024-05-09T20:16:50.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.472+0000 7fdc882e58c0 1 < canceled part key=_multipart_OGVA2778MJWU.2~8ME0VILLXX4A.0 size=1048576 2024-05-09T20:16:50.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.472+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~8ME0VILLXX4A.1 size=1048576 2024-05-09T20:16:50.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.476+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~8ME0VILLXX4A.2 size=909472 2024-05-09T20:16:50.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.484+0000 7fdc882e58c0 1 < canceled multipart upload key=OGVA2778MJWU upload=8ME0VILLXX4A tag=6B35HIESB5FY type=write size=4055200 2024-05-09T20:16:50.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.484+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=L397QXJVYLWQ type=del size=2490456 2024-05-09T20:16:50.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.488+0000 7fdc882e58c0 1 < canceled operation key=OGVA2778MJWU tag=KDJYYL8Y41GE type=del size=2483521 2024-05-09T20:16:50.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.496+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~23BQER4DXH65.0 size=1048576 2024-05-09T20:16:50.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.500+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~23BQER4DXH65.1 size=1048576 2024-05-09T20:16:50.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.500+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~23BQER4DXH65.2 size=93799 2024-05-09T20:16:50.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.500+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=23BQER4DXH65 tag=VMLF432UH20O type=write size=3239527 2024-05-09T20:16:50.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.504+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=JY493HNDYJBI type=del 2024-05-09T20:16:50.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.508+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=YXQU7UW3ELUH type=del size=963801 2024-05-09T20:16:50.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.512+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=T7GF3NPPD5LH type=del 2024-05-09T20:16:50.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.516+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=3W0RZ9MABRLX type=del size=2347850 2024-05-09T20:16:50.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.520+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=BS80R9OTICFH type=del 2024-05-09T20:16:50.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.524+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=TQE90Q0TLFUN type=del size=746613 2024-05-09T20:16:50.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.528+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=4G5OHXSPRCMZ type=del 2024-05-09T20:16:50.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.532+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=WBXS3E34OP09 type=del size=1217377 2024-05-09T20:16:50.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.536+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~LDTCM3HINYUB.0 size=917695 2024-05-09T20:16:50.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.540+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=LDTCM3HINYUB tag=S4PXJD7E16VH type=write size=1966271 2024-05-09T20:16:50.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.544+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=Y24D0SUYHFS5 type=write size=15870 2024-05-09T20:16:50.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.548+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=R0R2359Z8UDY type=del 2024-05-09T20:16:50.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.552+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=K1IIF3RSU1XP type=del size=1186712 2024-05-09T20:16:50.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.552+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=A74MUWV4SBLA type=del 2024-05-09T20:16:50.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.556+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=MSDUO7ZMZO3M type=del size=2505548 2024-05-09T20:16:50.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.560+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=CAKCMX71Z8EE type=del 2024-05-09T20:16:50.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.568+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~0EMVU3T7D0GZ.0 size=283084 2024-05-09T20:16:50.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.568+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=0EMVU3T7D0GZ tag=C45MCWFMDQIJ type=write size=1331660 2024-05-09T20:16:50.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.572+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~6U1ZIIQGYSV6.0 size=829681 2024-05-09T20:16:50.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.576+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=6U1ZIIQGYSV6 tag=B6RRR9GGZVC4 type=write size=1878257 2024-05-09T20:16:50.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.584+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~VF74IN60X11X.0 size=1048576 2024-05-09T20:16:50.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.584+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~VF74IN60X11X.1 size=1048576 2024-05-09T20:16:50.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~VF74IN60X11X.2 size=349708 2024-05-09T20:16:50.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.588+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=VF74IN60X11X tag=IKK4D8QA03US type=write size=3495436 2024-05-09T20:16:50.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.592+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~0BBCSZ06WUB9.0 size=1048576 2024-05-09T20:16:50.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.596+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~0BBCSZ06WUB9.1 size=821210 2024-05-09T20:16:50.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.600+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=0BBCSZ06WUB9 tag=IEBKCMB8IRFJ type=write size=2918362 2024-05-09T20:16:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.604+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=R6T004YG22SQ type=write size=346909 2024-05-09T20:16:50.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.608+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~PAKR9N63VG9K.0 size=255297 2024-05-09T20:16:50.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.612+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=PAKR9N63VG9K tag=B6LIEQT32EJU type=write size=1303873 2024-05-09T20:16:50.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.616+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=HX9YTFT37O4W type=del size=1674322 2024-05-09T20:16:50.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.616+0000 7fdc882e58c0 1 < canceled part key=_multipart_O07UHAIJZF0R.2~6AGJ9TPX3US5.0 size=180692 2024-05-09T20:16:50.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.624+0000 7fdc882e58c0 1 < canceled multipart upload key=O07UHAIJZF0R upload=6AGJ9TPX3US5 tag=M3VFO6UBNNHW type=write size=1229268 2024-05-09T20:16:50.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.628+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~9CTXCS8V1CIY.0 size=533214 2024-05-09T20:16:50.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.628+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=9CTXCS8V1CIY tag=FTFANPEJU2M7 type=write size=1581790 2024-05-09T20:16:50.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.632+0000 7fdc882e58c0 1 < canceled operation key=TQ5ZHATLNVPP tag=9UIGU3ZLIR56 type=del size=463330 2024-05-09T20:16:50.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.636+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=1DQ6AVA3RF11 type=write size=589322 2024-05-09T20:16:50.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.640+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=OYIQFUN8MVLT type=del 2024-05-09T20:16:50.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.644+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=72I1IOB49N41 type=write size=1016936 2024-05-09T20:16:50.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.648+0000 7fdc882e58c0 1 < canceled operation key=5NEOR8OOFAFZ tag=L397QXJVYLWQ type=del size=2490456 2024-05-09T20:16:50.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.656+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~YPUAZ7G93N2K.0 size=1048576 2024-05-09T20:16:50.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.656+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~YPUAZ7G93N2K.1 size=555773 2024-05-09T20:16:50.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.660+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=YPUAZ7G93N2K tag=XJCAFQIMT86N type=write size=2652925 2024-05-09T20:16:50.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.660+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~23BQER4DXH65.0 size=1048576 2024-05-09T20:16:50.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.664+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~23BQER4DXH65.1 size=1048576 2024-05-09T20:16:50.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.668+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~23BQER4DXH65.2 size=93799 2024-05-09T20:16:50.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.672+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=23BQER4DXH65 tag=VMLF432UH20O type=write size=3239527 2024-05-09T20:16:50.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.680+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~FLRS1EX5DJND.0 size=1048576 2024-05-09T20:16:50.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.684+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~FLRS1EX5DJND.1 size=151996 2024-05-09T20:16:50.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.684+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=FLRS1EX5DJND tag=AT1YH8A2ROY0 type=write size=2249148 2024-05-09T20:16:50.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.684+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=YXQU7UW3ELUH type=del 2024-05-09T20:16:50.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.688+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=MC3KXU2UE80Y type=del size=1835468 2024-05-09T20:16:50.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.692+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=3W0RZ9MABRLX type=del 2024-05-09T20:16:50.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.700+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~JMJAVDT2U2VY.0 size=1048576 2024-05-09T20:16:50.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.704+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~JMJAVDT2U2VY.1 size=757360 2024-05-09T20:16:50.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.704+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=JMJAVDT2U2VY tag=2N13DRW6AEXF type=write size=2854512 2024-05-09T20:16:50.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.704+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=TQE90Q0TLFUN type=del 2024-05-09T20:16:50.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.708+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=H51DDLSY6XT5 type=del size=1179121 2024-05-09T20:16:50.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.712+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=WBXS3E34OP09 type=del 2024-05-09T20:16:50.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.716+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=CZM0XZUIEENV type=del size=4065360 2024-05-09T20:16:50.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.720+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=Y24D0SUYHFS5 type=write size=15870 2024-05-09T20:16:50.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.724+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=OTEPF010I8R9 type=del size=1771130 2024-05-09T20:16:50.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.732+0000 7fdc882e58c0 1 < suggested remove operation key=SPXHCWZICCWY tag=K1IIF3RSU1XP type=del 2024-05-09T20:16:50.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~7NKDP54EAXZJ.0 size=1048576 2024-05-09T20:16:50.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.740+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~7NKDP54EAXZJ.1 size=743803 2024-05-09T20:16:50.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.740+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=7NKDP54EAXZJ tag=AGMSH414VHV5 type=write size=2840955 2024-05-09T20:16:50.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.744+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=MSDUO7ZMZO3M type=del 2024-05-09T20:16:50.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.748+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=F9I909VL7I7X type=del size=2531804 2024-05-09T20:16:50.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.748+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~0EMVU3T7D0GZ.0 size=283084 2024-05-09T20:16:50.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.756+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=0EMVU3T7D0GZ tag=C45MCWFMDQIJ type=write size=1331660 2024-05-09T20:16:50.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.760+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=MHASH8ZK3JSD type=write size=39948 2024-05-09T20:16:50.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.764+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~VF74IN60X11X.0 size=1048576 2024-05-09T20:16:50.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.764+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~VF74IN60X11X.1 size=1048576 2024-05-09T20:16:50.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.768+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~VF74IN60X11X.2 size=349708 2024-05-09T20:16:50.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.772+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=VF74IN60X11X tag=IKK4D8QA03US type=write size=3495436 2024-05-09T20:16:50.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~K5I70584IPSG.0 size=1048576 2024-05-09T20:16:50.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~K5I70584IPSG.1 size=1048576 2024-05-09T20:16:50.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~K5I70584IPSG.2 size=332944 2024-05-09T20:16:50.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.784+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=K5I70584IPSG tag=C8IIBIZUW0YJ type=write size=3478672 2024-05-09T20:16:50.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.792+0000 7fdc882e58c0 1 < suggested update operation key=EIMPFFK4I6NZ tag=R6T004YG22SQ type=write size=346909 2024-05-09T20:16:50.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.800+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~AXL3FF6VU7XL.0 size=1048576 2024-05-09T20:16:50.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.800+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~AXL3FF6VU7XL.1 size=746818 2024-05-09T20:16:50.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.800+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=AXL3FF6VU7XL tag=OISI0QIR8O9T type=write size=2843970 2024-05-09T20:16:50.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.804+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=HX9YTFT37O4W type=del 2024-05-09T20:16:50.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.808+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=SY19Y5O57NW6 type=del size=2482555 2024-05-09T20:16:50.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.812+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~9CTXCS8V1CIY.0 size=533214 2024-05-09T20:16:50.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.816+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=9CTXCS8V1CIY tag=FTFANPEJU2M7 type=write size=1581790 2024-05-09T20:16:50.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.824+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~F05MWAP5H4L1.0 size=1048576 2024-05-09T20:16:50.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.824+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~F05MWAP5H4L1.1 size=641743 2024-05-09T20:16:50.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.824+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=F05MWAP5H4L1 tag=YDSW7XTPRLTL type=write size=2738895 2024-05-09T20:16:50.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.828+0000 7fdc882e58c0 1 < completed write operation key=NNGWUSCETMH4 tag=1DQ6AVA3RF11 type=write size=589322 2024-05-09T20:16:50.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.836+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~2FTZL4VHATXE.0 size=1048576 2024-05-09T20:16:50.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.836+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~2FTZL4VHATXE.1 size=445014 2024-05-09T20:16:50.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.836+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=2FTZL4VHATXE tag=O1I0YLFX4WCL type=write size=2542166 2024-05-09T20:16:50.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.840+0000 7fdc882e58c0 1 < completed write operation key=QM53IENH9CX4 tag=72I1IOB49N41 type=write size=1016936 2024-05-09T20:16:50.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.844+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=5OI4072SF72K type=write size=654683 2024-05-09T20:16:50.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.848+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~YPUAZ7G93N2K.0 size=1048576 2024-05-09T20:16:50.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.848+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~YPUAZ7G93N2K.1 size=555773 2024-05-09T20:16:50.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.852+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=YPUAZ7G93N2K tag=XJCAFQIMT86N type=write size=2652925 2024-05-09T20:16:50.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.856+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=JOHPE7SNHALV type=write size=189701 2024-05-09T20:16:50.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.860+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~FLRS1EX5DJND.0 size=1048576 2024-05-09T20:16:50.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.864+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~FLRS1EX5DJND.1 size=151996 2024-05-09T20:16:50.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.868+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=FLRS1EX5DJND tag=AT1YH8A2ROY0 type=write size=2249148 2024-05-09T20:16:50.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.872+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=OLCJD5X2YIVW type=write size=861980 2024-05-09T20:16:50.871 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.880+0000 7fdc882e58c0 1 < suggested remove operation key=WC5YH0OU51WX tag=MC3KXU2UE80Y type=del 2024-05-09T20:16:50.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.884+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=3ENO4GM5B55J type=del size=806443 2024-05-09T20:16:50.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.888+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~JMJAVDT2U2VY.0 size=1048576 2024-05-09T20:16:50.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.888+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~JMJAVDT2U2VY.1 size=757360 2024-05-09T20:16:50.886 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.896+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=JMJAVDT2U2VY tag=2N13DRW6AEXF type=write size=2854512 2024-05-09T20:16:50.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~PTU43S4ZC1BS.0 size=1048576 2024-05-09T20:16:50.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.904+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~PTU43S4ZC1BS.1 size=1048576 2024-05-09T20:16:50.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.908+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~PTU43S4ZC1BS.2 size=354640 2024-05-09T20:16:50.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.908+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=PTU43S4ZC1BS tag=APH114AFG18Z type=write size=3500368 2024-05-09T20:16:50.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.908+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=H51DDLSY6XT5 type=del 2024-05-09T20:16:50.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.912+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=896F9YLQVC7K type=del size=2321927 2024-05-09T20:16:50.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.916+0000 7fdc882e58c0 1 < canceled operation key=QM53IENH9CX4 tag=CZM0XZUIEENV type=del size=4065360 2024-05-09T20:16:50.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.924+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~PZFAF0EGAAZY.0 size=788310 2024-05-09T20:16:50.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.924+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=PZFAF0EGAAZY tag=X0O8ER5T9MXB type=write size=1836886 2024-05-09T20:16:50.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.928+0000 7fdc882e58c0 1 < suggested remove operation key=P2DNA4DGI54P tag=OTEPF010I8R9 type=del 2024-05-09T20:16:50.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~U9B2E1ZLKAKY.0 size=1048576 2024-05-09T20:16:50.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~U9B2E1ZLKAKY.1 size=344393 2024-05-09T20:16:50.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.936+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=U9B2E1ZLKAKY tag=O0A4QMCH1OAL type=write size=2441545 2024-05-09T20:16:50.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.940+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~7NKDP54EAXZJ.0 size=1048576 2024-05-09T20:16:50.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.944+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~7NKDP54EAXZJ.1 size=743803 2024-05-09T20:16:50.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.948+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=7NKDP54EAXZJ tag=AGMSH414VHV5 type=write size=2840955 2024-05-09T20:16:50.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.952+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=Z4Z6WM0B8AAS type=del size=427192 2024-05-09T20:16:50.946 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.956+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=F9I909VL7I7X type=del 2024-05-09T20:16:50.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.960+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~B2QS7349JL7B.0 size=1048576 2024-05-09T20:16:50.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.964+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~B2QS7349JL7B.1 size=1048576 2024-05-09T20:16:50.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.964+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~B2QS7349JL7B.2 size=644257 2024-05-09T20:16:50.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.964+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=B2QS7349JL7B tag=AJ9KK3YVF7S9 type=write size=3789985 2024-05-09T20:16:50.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.972+0000 7fdc882e58c0 1 < suggested update operation key=AJANS5ADJO5B tag=MHASH8ZK3JSD type=write size=39948 2024-05-09T20:16:50.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.972+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=D6EMJJ2T0VQJ type=del size=3429719 2024-05-09T20:16:50.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.976+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~K5I70584IPSG.0 size=1048576 2024-05-09T20:16:50.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.980+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~K5I70584IPSG.1 size=1048576 2024-05-09T20:16:50.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.980+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~K5I70584IPSG.2 size=332944 2024-05-09T20:16:50.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.988+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=K5I70584IPSG tag=C8IIBIZUW0YJ type=write size=3478672 2024-05-09T20:16:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.992+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=LVRGKK5QKNU0 type=write size=337073 2024-05-09T20:16:50.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.992+0000 7fdc882e58c0 1 < canceled part key=_multipart_NYHR6GTPQYZS.2~AXL3FF6VU7XL.0 size=1048576 2024-05-09T20:16:50.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:50.996+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~AXL3FF6VU7XL.1 size=746818 2024-05-09T20:16:50.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.000+0000 7fdc882e58c0 1 < canceled multipart upload key=NYHR6GTPQYZS upload=AXL3FF6VU7XL tag=OISI0QIR8O9T type=write size=2843970 2024-05-09T20:16:50.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.004+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=59L261KI4LMB type=write size=84645 2024-05-09T20:16:50.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.008+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=SY19Y5O57NW6 type=del 2024-05-09T20:16:51.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.012+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~1WYK6TJXVD8T.0 size=1048576 2024-05-09T20:16:51.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~1WYK6TJXVD8T.1 size=1048576 2024-05-09T20:16:51.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~1WYK6TJXVD8T.2 size=833785 2024-05-09T20:16:51.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.016+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=1WYK6TJXVD8T tag=8FEG7FQKQ70J type=write size=3979513 2024-05-09T20:16:51.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.020+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~F05MWAP5H4L1.0 size=1048576 2024-05-09T20:16:51.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.024+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~F05MWAP5H4L1.1 size=641743 2024-05-09T20:16:51.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.028+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=F05MWAP5H4L1 tag=YDSW7XTPRLTL type=write size=2738895 2024-05-09T20:16:51.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.032+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=FQI4MGW6MU6M type=del size=3050361 2024-05-09T20:16:51.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.036+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~2FTZL4VHATXE.0 size=1048576 2024-05-09T20:16:51.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.040+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~2FTZL4VHATXE.1 size=445014 2024-05-09T20:16:51.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.044+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=2FTZL4VHATXE tag=O1I0YLFX4WCL type=write size=2542166 2024-05-09T20:16:51.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.048+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=JUYLIDW0FXU3 type=del size=205552 2024-05-09T20:16:51.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.052+0000 7fdc882e58c0 1 < canceled operation key=SPXHCWZICCWY tag=5OI4072SF72K type=write size=654683 2024-05-09T20:16:51.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.060+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~N208RE17N0AP.0 size=1048576 2024-05-09T20:16:51.054 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.064+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~N208RE17N0AP.1 size=701216 2024-05-09T20:16:51.054 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.064+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=N208RE17N0AP tag=WRF6SCU28TXK type=write size=2798368 2024-05-09T20:16:51.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.064+0000 7fdc882e58c0 1 < completed write operation key=SPXHCWZICCWY tag=JOHPE7SNHALV type=write size=189701 2024-05-09T20:16:51.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.068+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=IJMPAT3MGQQ9 type=del size=1439915 2024-05-09T20:16:51.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.072+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=OLCJD5X2YIVW type=write size=861980 2024-05-09T20:16:51.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.080+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~ZQTJUMRV536M.0 size=1048576 2024-05-09T20:16:51.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.084+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~ZQTJUMRV536M.1 size=1048576 2024-05-09T20:16:51.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.084+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~ZQTJUMRV536M.2 size=943382 2024-05-09T20:16:51.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.084+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=ZQTJUMRV536M tag=38F3N6DNK34B type=write size=4089110 2024-05-09T20:16:51.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.088+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=3ENO4GM5B55J type=del 2024-05-09T20:16:51.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.096+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~IQ13GDQ1T1R6.0 size=1048576 2024-05-09T20:16:51.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.096+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~IQ13GDQ1T1R6.1 size=627024 2024-05-09T20:16:51.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.096+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=IQ13GDQ1T1R6 tag=BVZQVX6YWRPU type=write size=2724176 2024-05-09T20:16:51.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.100+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~PTU43S4ZC1BS.0 size=1048576 2024-05-09T20:16:51.095 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.104+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~PTU43S4ZC1BS.1 size=1048576 2024-05-09T20:16:51.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.108+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~PTU43S4ZC1BS.2 size=354640 2024-05-09T20:16:51.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.112+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=PTU43S4ZC1BS tag=APH114AFG18Z type=write size=3500368 2024-05-09T20:16:51.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.116+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~FK21U0EQQE1Y.0 size=1048576 2024-05-09T20:16:51.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.120+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~FK21U0EQQE1Y.1 size=637087 2024-05-09T20:16:51.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.120+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=FK21U0EQQE1Y tag=KQN5F9HMTPIJ type=write size=2734239 2024-05-09T20:16:51.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.120+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=896F9YLQVC7K type=del 2024-05-09T20:16:51.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.124+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=XZU97PUY9RWR type=del size=3100002 2024-05-09T20:16:51.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.128+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~PZFAF0EGAAZY.0 size=788310 2024-05-09T20:16:51.124 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.132+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=PZFAF0EGAAZY tag=X0O8ER5T9MXB type=write size=1836886 2024-05-09T20:16:51.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.136+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=867953P8IQUH type=del size=1034015 2024-05-09T20:16:51.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.136+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~U9B2E1ZLKAKY.0 size=1048576 2024-05-09T20:16:51.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.140+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~U9B2E1ZLKAKY.1 size=344393 2024-05-09T20:16:51.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.148+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=U9B2E1ZLKAKY tag=O0A4QMCH1OAL type=write size=2441545 2024-05-09T20:16:51.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.152+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=9JQGOK30YZE5 type=write size=398223 2024-05-09T20:16:51.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.152+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=Z4Z6WM0B8AAS type=del 2024-05-09T20:16:51.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.160+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~1CZTRSSUWEMG.0 size=1048576 2024-05-09T20:16:51.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.160+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~1CZTRSSUWEMG.1 size=1048576 2024-05-09T20:16:51.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.164+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~1CZTRSSUWEMG.2 size=163704 2024-05-09T20:16:51.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.164+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=1CZTRSSUWEMG tag=7AACD2GDAIS4 type=write size=3309432 2024-05-09T20:16:51.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.168+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~B2QS7349JL7B.0 size=1048576 2024-05-09T20:16:51.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.168+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~B2QS7349JL7B.1 size=1048576 2024-05-09T20:16:51.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.172+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~B2QS7349JL7B.2 size=644257 2024-05-09T20:16:51.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.176+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=B2QS7349JL7B tag=AJ9KK3YVF7S9 type=write size=3789985 2024-05-09T20:16:51.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.180+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~HNDLHNPY6U05.0 size=758402 2024-05-09T20:16:51.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.180+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=HNDLHNPY6U05 tag=E57X1R3WSJ1K type=write size=1806978 2024-05-09T20:16:51.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.184+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=D6EMJJ2T0VQJ type=del 2024-05-09T20:16:51.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.188+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=7734EG9BZYQJ type=del size=2899242 2024-05-09T20:16:51.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.192+0000 7fdc882e58c0 1 < completed write operation key=SNPNT5JFJOUS tag=LVRGKK5QKNU0 type=write size=337073 2024-05-09T20:16:51.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.196+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~GBFRX724FF2L.0 size=1048576 2024-05-09T20:16:51.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.200+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~GBFRX724FF2L.1 size=842749 2024-05-09T20:16:51.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.200+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=GBFRX724FF2L tag=O34YAOFO96YP type=write size=2939901 2024-05-09T20:16:51.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.204+0000 7fdc882e58c0 1 < completed write operation key=NBAECV9L85UK tag=59L261KI4LMB type=write size=84645 2024-05-09T20:16:51.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.208+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=VSPNPPTC4OOY type=del size=1042918 2024-05-09T20:16:51.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.212+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~1WYK6TJXVD8T.0 size=1048576 2024-05-09T20:16:51.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.212+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~1WYK6TJXVD8T.1 size=1048576 2024-05-09T20:16:51.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.212+0000 7fdc882e58c0 1 < canceled part key=_multipart_54R9XPO62J0M.2~1WYK6TJXVD8T.2 size=833785 2024-05-09T20:16:51.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.216+0000 7fdc882e58c0 1 < canceled multipart upload key=54R9XPO62J0M upload=1WYK6TJXVD8T tag=8FEG7FQKQ70J type=write size=3979513 2024-05-09T20:16:51.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.224+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~VT2FFN05VW45.0 size=1048576 2024-05-09T20:16:51.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.224+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~VT2FFN05VW45.1 size=1048576 2024-05-09T20:16:51.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.228+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~VT2FFN05VW45.2 size=245933 2024-05-09T20:16:51.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.228+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=VT2FFN05VW45 tag=535GPXV55R9N type=write size=3391661 2024-05-09T20:16:51.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.236+0000 7fdc882e58c0 1 < suggested update operation key=EIMPFFK4I6NZ tag=FQI4MGW6MU6M type=del size=3050361 2024-05-09T20:16:51.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.236+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=I97IBM11XNL2 type=write size=808081 2024-05-09T20:16:51.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.240+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=JUYLIDW0FXU3 type=del 2024-05-09T20:16:51.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.248+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~D5S37JW34FGQ.0 size=1048576 2024-05-09T20:16:51.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.248+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~D5S37JW34FGQ.1 size=183571 2024-05-09T20:16:51.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.248+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=D5S37JW34FGQ tag=G2HAKA2XH5DX type=write size=2280723 2024-05-09T20:16:51.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.252+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~N208RE17N0AP.0 size=1048576 2024-05-09T20:16:51.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.256+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~N208RE17N0AP.1 size=701216 2024-05-09T20:16:51.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.260+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=N208RE17N0AP tag=WRF6SCU28TXK type=write size=2798368 2024-05-09T20:16:51.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.268+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~Q1O9X3KLFLPU.0 size=117508 2024-05-09T20:16:51.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.268+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=Q1O9X3KLFLPU tag=1F5HXQRE9YLZ type=write size=1166084 2024-05-09T20:16:51.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.272+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=IJMPAT3MGQQ9 type=del 2024-05-09T20:16:51.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.276+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~S840U6VW18R3.0 size=1048576 2024-05-09T20:16:51.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.280+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~S840U6VW18R3.1 size=619216 2024-05-09T20:16:51.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.280+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=S840U6VW18R3 tag=EK8P0P0HIO9C type=write size=2716368 2024-05-09T20:16:51.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.284+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~ZQTJUMRV536M.0 size=1048576 2024-05-09T20:16:51.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.284+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~ZQTJUMRV536M.1 size=1048576 2024-05-09T20:16:51.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.284+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~ZQTJUMRV536M.2 size=943382 2024-05-09T20:16:51.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.292+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=ZQTJUMRV536M tag=38F3N6DNK34B type=write size=4089110 2024-05-09T20:16:51.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.292+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=YXFIP505EKMJ type=del size=597478 2024-05-09T20:16:51.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.296+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~IQ13GDQ1T1R6.0 size=1048576 2024-05-09T20:16:51.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.300+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~IQ13GDQ1T1R6.1 size=627024 2024-05-09T20:16:51.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.304+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=IQ13GDQ1T1R6 tag=BVZQVX6YWRPU type=write size=2724176 2024-05-09T20:16:51.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.308+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~GZYUUM9508IN.0 size=745655 2024-05-09T20:16:51.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.308+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=GZYUUM9508IN tag=N6MUPAKO6G6I type=write size=1794231 2024-05-09T20:16:51.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.312+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~FK21U0EQQE1Y.0 size=1048576 2024-05-09T20:16:51.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.316+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~FK21U0EQQE1Y.1 size=637087 2024-05-09T20:16:51.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.320+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=FK21U0EQQE1Y tag=KQN5F9HMTPIJ type=write size=2734239 2024-05-09T20:16:51.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.328+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~LVJQDU6ZSKQM.0 size=1048576 2024-05-09T20:16:51.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.328+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~LVJQDU6ZSKQM.1 size=1048576 2024-05-09T20:16:51.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.332+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~LVJQDU6ZSKQM.2 size=1028142 2024-05-09T20:16:51.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.332+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=LVJQDU6ZSKQM tag=EHOSMFGGQPMM type=write size=4173870 2024-05-09T20:16:51.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.336+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=XZU97PUY9RWR type=del 2024-05-09T20:16:51.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.336+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=FJ2RBYR9TNZW type=del size=1893429 2024-05-09T20:16:51.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.344+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=867953P8IQUH type=del 2024-05-09T20:16:51.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.348+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~VJ7O4T3CGNAY.0 size=1048576 2024-05-09T20:16:51.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.352+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~VJ7O4T3CGNAY.1 size=718959 2024-05-09T20:16:51.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.352+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=VJ7O4T3CGNAY tag=NB38BQ2M65LX type=write size=2816111 2024-05-09T20:16:51.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.356+0000 7fdc882e58c0 1 < completed write operation key=IZPPFLWVPRKO tag=9JQGOK30YZE5 type=write size=398223 2024-05-09T20:16:51.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.360+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=ERWG02LN5V16 type=del size=3337335 2024-05-09T20:16:51.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.364+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~1CZTRSSUWEMG.0 size=1048576 2024-05-09T20:16:51.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.364+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~1CZTRSSUWEMG.1 size=1048576 2024-05-09T20:16:51.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.368+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~1CZTRSSUWEMG.2 size=163704 2024-05-09T20:16:51.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.372+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=1CZTRSSUWEMG tag=7AACD2GDAIS4 type=write size=3309432 2024-05-09T20:16:51.366 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.376+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=ZTR7QZLGNI22 type=del size=353478 2024-05-09T20:16:51.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.376+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~HNDLHNPY6U05.0 size=758402 2024-05-09T20:16:51.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.384+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=HNDLHNPY6U05 tag=E57X1R3WSJ1K type=write size=1806978 2024-05-09T20:16:51.377 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.384+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=COMA0YNP46FF type=del size=3997291 2024-05-09T20:16:51.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.392+0000 7fdc882e58c0 1 < suggested remove operation key=T6T3IOW9VPA0 tag=7734EG9BZYQJ type=del 2024-05-09T20:16:51.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.396+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=0L5NRCGM2ZIY type=del size=4181203 2024-05-09T20:16:51.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.396+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~GBFRX724FF2L.0 size=1048576 2024-05-09T20:16:51.391 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.400+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~GBFRX724FF2L.1 size=842749 2024-05-09T20:16:51.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.404+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=GBFRX724FF2L tag=O34YAOFO96YP type=write size=2939901 2024-05-09T20:16:51.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.408+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=VZ298B8IEDTH type=write size=135898 2024-05-09T20:16:51.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.412+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=VSPNPPTC4OOY type=del 2024-05-09T20:16:51.406 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.416+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=U4KGIQ5C3ORO type=del size=3445134 2024-05-09T20:16:51.406 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.416+0000 7fdc882e58c0 1 < canceled part key=_multipart_FNB9B18LIA8L.2~VT2FFN05VW45.0 size=1048576 2024-05-09T20:16:51.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.416+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~VT2FFN05VW45.1 size=1048576 2024-05-09T20:16:51.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.416+0000 7fdc882e58c0 1 < canceled part key=_multipart_FNB9B18LIA8L.2~VT2FFN05VW45.2 size=245933 2024-05-09T20:16:51.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.424+0000 7fdc882e58c0 1 < canceled multipart upload key=FNB9B18LIA8L upload=VT2FFN05VW45 tag=535GPXV55R9N type=write size=3391661 2024-05-09T20:16:51.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.428+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~VK0BZFX6373U.0 size=447414 2024-05-09T20:16:51.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.428+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=VK0BZFX6373U tag=UEWMJ3OYSP5G type=write size=1495990 2024-05-09T20:16:51.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.432+0000 7fdc882e58c0 1 < completed write operation key=O07UHAIJZF0R tag=I97IBM11XNL2 type=write size=808081 2024-05-09T20:16:51.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.436+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~LEDY0L9RNKTX.0 size=173770 2024-05-09T20:16:51.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.436+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=LEDY0L9RNKTX tag=YOYMPYZB1GCN type=write size=1222346 2024-05-09T20:16:51.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.440+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~D5S37JW34FGQ.0 size=1048576 2024-05-09T20:16:51.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.444+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~D5S37JW34FGQ.1 size=183571 2024-05-09T20:16:51.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.448+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=D5S37JW34FGQ tag=G2HAKA2XH5DX type=write size=2280723 2024-05-09T20:16:51.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.452+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=CKQIRX5BL4LY type=write size=401005 2024-05-09T20:16:51.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.452+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~Q1O9X3KLFLPU.0 size=117508 2024-05-09T20:16:51.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.460+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=Q1O9X3KLFLPU tag=1F5HXQRE9YLZ type=write size=1166084 2024-05-09T20:16:51.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.468+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~UVG7BFSS1IST.0 size=634941 2024-05-09T20:16:51.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.468+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=UVG7BFSS1IST tag=LM14MUWR4JH1 type=write size=1683517 2024-05-09T20:16:51.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.468+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~S840U6VW18R3.0 size=1048576 2024-05-09T20:16:51.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.472+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~S840U6VW18R3.1 size=619216 2024-05-09T20:16:51.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.476+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=S840U6VW18R3 tag=EK8P0P0HIO9C type=write size=2716368 2024-05-09T20:16:51.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.480+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=36YYBZ1TKGRF type=write size=479980 2024-05-09T20:16:51.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.484+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=YXFIP505EKMJ type=del 2024-05-09T20:16:51.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.488+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=QUZWIWQGMO1H type=write size=623001 2024-05-09T20:16:51.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.492+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~GZYUUM9508IN.0 size=745655 2024-05-09T20:16:51.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.496+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=GZYUUM9508IN tag=N6MUPAKO6G6I type=write size=1794231 2024-05-09T20:16:51.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.500+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~U0EFLCPC69EP.0 size=399555 2024-05-09T20:16:51.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.500+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=U0EFLCPC69EP tag=QPI8C2JA00X0 type=write size=1448131 2024-05-09T20:16:51.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.504+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~LVJQDU6ZSKQM.0 size=1048576 2024-05-09T20:16:51.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.504+0000 7fdc882e58c0 1 < canceled part key=_multipart_SPXHCWZICCWY.2~LVJQDU6ZSKQM.1 size=1048576 2024-05-09T20:16:51.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.508+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~LVJQDU6ZSKQM.2 size=1028142 2024-05-09T20:16:51.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.512+0000 7fdc882e58c0 1 < canceled multipart upload key=SPXHCWZICCWY upload=LVJQDU6ZSKQM tag=EHOSMFGGQPMM type=write size=4173870 2024-05-09T20:16:51.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.516+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=Z5BN1T60SOV6 type=del size=1385015 2024-05-09T20:16:51.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.520+0000 7fdc882e58c0 1 < canceled operation key=EK78CNKUHNKO tag=FJ2RBYR9TNZW type=del size=1893429 2024-05-09T20:16:51.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.524+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=62XQRYVTSEUW type=del size=315458 2024-05-09T20:16:51.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.528+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~VJ7O4T3CGNAY.0 size=1048576 2024-05-09T20:16:51.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.528+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~VJ7O4T3CGNAY.1 size=718959 2024-05-09T20:16:51.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.536+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=VJ7O4T3CGNAY tag=NB38BQ2M65LX type=write size=2816111 2024-05-09T20:16:51.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.540+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~WCTP0IBT0MD2.0 size=1048576 2024-05-09T20:16:51.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.544+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~WCTP0IBT0MD2.1 size=1048576 2024-05-09T20:16:51.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.548+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~WCTP0IBT0MD2.2 size=228065 2024-05-09T20:16:51.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.548+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=WCTP0IBT0MD2 tag=C2MWT6QWJP57 type=write size=3373793 2024-05-09T20:16:51.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.548+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=ERWG02LN5V16 type=del 2024-05-09T20:16:51.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.552+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=FJOBUT7PH303 type=del size=1221289 2024-05-09T20:16:51.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.560+0000 7fdc882e58c0 1 < suggested remove operation key=TQ5ZHATLNVPP tag=ZTR7QZLGNI22 type=del 2024-05-09T20:16:51.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.564+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=AG024PS3RH5E type=del size=1492400 2024-05-09T20:16:51.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.568+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=COMA0YNP46FF type=del 2024-05-09T20:16:51.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.572+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~ZY9WJW6KIC36.0 size=1048576 2024-05-09T20:16:51.568 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.576+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~ZY9WJW6KIC36.1 size=1048576 2024-05-09T20:16:51.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.580+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~ZY9WJW6KIC36.2 size=708387 2024-05-09T20:16:51.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.580+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=ZY9WJW6KIC36 tag=QVLW4CJOFCEP type=write size=3854115 2024-05-09T20:16:51.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.584+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=0L5NRCGM2ZIY type=del 2024-05-09T20:16:51.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~A15NTBG05P12.0 size=99692 2024-05-09T20:16:51.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.588+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=A15NTBG05P12 tag=2VEMVEC291UW type=write size=1148268 2024-05-09T20:16:51.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.596+0000 7fdc882e58c0 1 < suggested remove operation key=JS90W6PIREMD tag=VZ298B8IEDTH type=write 2024-05-09T20:16:51.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.600+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=9S6UXZ44R3GV type=del size=2757658 2024-05-09T20:16:51.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.604+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=U4KGIQ5C3ORO type=del 2024-05-09T20:16:51.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~ITLQCCWE7RNI.0 size=788804 2024-05-09T20:16:51.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.612+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=ITLQCCWE7RNI tag=MLQHXF1H5992 type=write size=1837380 2024-05-09T20:16:51.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.612+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~VK0BZFX6373U.0 size=447414 2024-05-09T20:16:51.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.620+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=VK0BZFX6373U tag=UEWMJ3OYSP5G type=write size=1495990 2024-05-09T20:16:51.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.624+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=OAM7D8GPH3JC type=write size=953733 2024-05-09T20:16:51.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.628+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~LEDY0L9RNKTX.0 size=173770 2024-05-09T20:16:51.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.632+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=LEDY0L9RNKTX tag=YOYMPYZB1GCN type=write size=1222346 2024-05-09T20:16:51.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.636+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=0QL2AIV3X1YF type=del size=595933 2024-05-09T20:16:51.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.644+0000 7fdc882e58c0 1 < suggested remove operation key=EK78CNKUHNKO tag=CKQIRX5BL4LY type=write 2024-05-09T20:16:51.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.652+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~WRXLVIMBWW13.0 size=1048576 2024-05-09T20:16:51.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.656+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~WRXLVIMBWW13.1 size=1048576 2024-05-09T20:16:51.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.656+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~WRXLVIMBWW13.2 size=794162 2024-05-09T20:16:51.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.656+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=WRXLVIMBWW13 tag=0HV4F7O7M3MR type=write size=3939890 2024-05-09T20:16:51.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.660+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~UVG7BFSS1IST.0 size=634941 2024-05-09T20:16:51.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.668+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=UVG7BFSS1IST tag=LM14MUWR4JH1 type=write size=1683517 2024-05-09T20:16:51.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.672+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=K0RZKQKVHZOD type=del size=3516475 2024-05-09T20:16:51.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.672+0000 7fdc882e58c0 1 < completed write operation key=CNV7NHP27JZ7 tag=36YYBZ1TKGRF type=write size=479980 2024-05-09T20:16:51.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.680+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=BQU328RFDPJ3 type=write size=119350 2024-05-09T20:16:51.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.680+0000 7fdc882e58c0 1 < completed write operation key=SPXHCWZICCWY tag=QUZWIWQGMO1H type=write size=623001 2024-05-09T20:16:51.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.684+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=6CYGVNJSQDWC type=del size=1120754 2024-05-09T20:16:51.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.688+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~U0EFLCPC69EP.0 size=399555 2024-05-09T20:16:51.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.696+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=U0EFLCPC69EP tag=QPI8C2JA00X0 type=write size=1448131 2024-05-09T20:16:51.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.700+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=ZGLAKIHYXUEA type=write size=313488 2024-05-09T20:16:51.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.704+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=Z5BN1T60SOV6 type=del 2024-05-09T20:16:51.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.708+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~NL843O5CK3OE.0 size=1048576 2024-05-09T20:16:51.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~NL843O5CK3OE.1 size=1048576 2024-05-09T20:16:51.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.716+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~NL843O5CK3OE.2 size=82429 2024-05-09T20:16:51.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.716+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=NL843O5CK3OE tag=BDRRFR0H55MJ type=write size=3228157 2024-05-09T20:16:51.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.720+0000 7fdc882e58c0 1 < suggested remove operation key=5UMJFJRKUPKR tag=62XQRYVTSEUW type=del 2024-05-09T20:16:51.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.724+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=ZRS62LQCPX4X type=del size=3063783 2024-05-09T20:16:51.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.728+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~WCTP0IBT0MD2.0 size=1048576 2024-05-09T20:16:51.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.732+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~WCTP0IBT0MD2.1 size=1048576 2024-05-09T20:16:51.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.736+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~WCTP0IBT0MD2.2 size=228065 2024-05-09T20:16:51.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.740+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=WCTP0IBT0MD2 tag=C2MWT6QWJP57 type=write size=3373793 2024-05-09T20:16:51.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.744+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=KA8SQZO32YJD type=del size=1117845 2024-05-09T20:16:51.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.748+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=FJOBUT7PH303 type=del 2024-05-09T20:16:51.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.756+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~ZDHPZBS3IHTH.0 size=1048576 2024-05-09T20:16:51.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.756+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~ZDHPZBS3IHTH.1 size=1048576 2024-05-09T20:16:51.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.760+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~ZDHPZBS3IHTH.2 size=637322 2024-05-09T20:16:51.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.760+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=ZDHPZBS3IHTH tag=P8Y46X9SD8BA type=write size=3783050 2024-05-09T20:16:51.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.764+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=AG024PS3RH5E type=del 2024-05-09T20:16:51.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.772+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~NYP4HRPAOKCU.0 size=743015 2024-05-09T20:16:51.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.772+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=NYP4HRPAOKCU tag=1JBXXOFN0S3H type=write size=1791591 2024-05-09T20:16:51.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.772+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~ZY9WJW6KIC36.0 size=1048576 2024-05-09T20:16:51.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.772+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~ZY9WJW6KIC36.1 size=1048576 2024-05-09T20:16:51.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.776+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~ZY9WJW6KIC36.2 size=708387 2024-05-09T20:16:51.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.784+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=ZY9WJW6KIC36 tag=QVLW4CJOFCEP type=write size=3854115 2024-05-09T20:16:51.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.788+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~BGQ0A48LDSCK.0 size=1048576 2024-05-09T20:16:51.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.792+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~BGQ0A48LDSCK.1 size=1048576 2024-05-09T20:16:51.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.796+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~BGQ0A48LDSCK.2 size=1037772 2024-05-09T20:16:51.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.796+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=BGQ0A48LDSCK tag=WZVHAYYHEHG1 type=write size=4183500 2024-05-09T20:16:51.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.800+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~A15NTBG05P12.0 size=99692 2024-05-09T20:16:51.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.804+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=A15NTBG05P12 tag=2VEMVEC291UW type=write size=1148268 2024-05-09T20:16:51.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.812+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~HUOHEUA1ERDO.0 size=1048576 2024-05-09T20:16:51.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.812+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~HUOHEUA1ERDO.1 size=1048576 2024-05-09T20:16:51.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.816+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~HUOHEUA1ERDO.2 size=192857 2024-05-09T20:16:51.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.816+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=HUOHEUA1ERDO tag=3ZLAZWLVX4IT type=write size=3338585 2024-05-09T20:16:51.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.820+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=9S6UXZ44R3GV type=del 2024-05-09T20:16:51.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.820+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=ILQDSPN3I9CX type=write size=505627 2024-05-09T20:16:51.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.824+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~ITLQCCWE7RNI.0 size=788804 2024-05-09T20:16:51.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.828+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=ITLQCCWE7RNI tag=MLQHXF1H5992 type=write size=1837380 2024-05-09T20:16:51.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.832+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=6K0QENK5PUP1 type=del size=1648551 2024-05-09T20:16:51.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.836+0000 7fdc882e58c0 1 < canceled operation key=T6T3IOW9VPA0 tag=OAM7D8GPH3JC type=write size=953733 2024-05-09T20:16:51.830 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.840+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=WNNNYYZAP5UO type=del size=535794 2024-05-09T20:16:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.840+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=0QL2AIV3X1YF type=del 2024-05-09T20:16:51.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.848+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~Q5VAP3WS7SI7.0 size=1048576 2024-05-09T20:16:51.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.848+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~Q5VAP3WS7SI7.1 size=195854 2024-05-09T20:16:51.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.848+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=Q5VAP3WS7SI7 tag=FB7Q9IC5100N type=write size=2293006 2024-05-09T20:16:51.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.852+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~WRXLVIMBWW13.0 size=1048576 2024-05-09T20:16:51.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.852+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~WRXLVIMBWW13.1 size=1048576 2024-05-09T20:16:51.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.856+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~WRXLVIMBWW13.2 size=794162 2024-05-09T20:16:51.853 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.860+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=WRXLVIMBWW13 tag=0HV4F7O7M3MR type=write size=3939890 2024-05-09T20:16:51.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.864+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=ONG3FL6D0NS4 type=del size=941867 2024-05-09T20:16:51.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.868+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=K0RZKQKVHZOD type=del 2024-05-09T20:16:51.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.872+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~5S5WA6FTKY5R.0 size=1048576 2024-05-09T20:16:51.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.876+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~5S5WA6FTKY5R.1 size=1048576 2024-05-09T20:16:51.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.876+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~5S5WA6FTKY5R.2 size=361103 2024-05-09T20:16:51.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.876+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=5S5WA6FTKY5R tag=FH3696QPS1DQ type=write size=3506831 2024-05-09T20:16:51.872 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.880+0000 7fdc882e58c0 1 < completed write operation key=346HV1XQKCOZ tag=BQU328RFDPJ3 type=write size=119350 2024-05-09T20:16:51.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.888+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~UCP9554R242F.0 size=1048576 2024-05-09T20:16:51.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.888+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~UCP9554R242F.1 size=737355 2024-05-09T20:16:51.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.888+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=UCP9554R242F tag=3D9EO6Y7QCFN type=write size=2834507 2024-05-09T20:16:51.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.892+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=6CYGVNJSQDWC type=del 2024-05-09T20:16:51.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.896+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~JPNQS6XQAH2N.0 size=1048576 2024-05-09T20:16:51.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~JPNQS6XQAH2N.1 size=1048576 2024-05-09T20:16:51.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~JPNQS6XQAH2N.2 size=197719 2024-05-09T20:16:51.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.900+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=JPNQS6XQAH2N tag=9RC0Z90BURDC type=write size=3343447 2024-05-09T20:16:51.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.904+0000 7fdc882e58c0 1 < completed write operation key=NYHR6GTPQYZS tag=ZGLAKIHYXUEA type=write size=313488 2024-05-09T20:16:51.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.908+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=MUH64S6GNNMB type=del size=4034613 2024-05-09T20:16:51.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.912+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~NL843O5CK3OE.0 size=1048576 2024-05-09T20:16:51.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.912+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~NL843O5CK3OE.1 size=1048576 2024-05-09T20:16:51.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.916+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~NL843O5CK3OE.2 size=82429 2024-05-09T20:16:51.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.920+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=NL843O5CK3OE tag=BDRRFR0H55MJ type=write size=3228157 2024-05-09T20:16:51.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.924+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=BGNL7SCM0U4U type=write size=923087 2024-05-09T20:16:51.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.928+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=ZRS62LQCPX4X type=del 2024-05-09T20:16:51.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.932+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~L8OYZAK9PPFZ.0 size=93225 2024-05-09T20:16:51.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.932+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=L8OYZAK9PPFZ tag=IBDDTGYNPOGL type=write size=1141801 2024-05-09T20:16:51.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.936+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=KA8SQZO32YJD type=del 2024-05-09T20:16:51.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.936+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=2ZNYAY5D7I9V type=del size=3895327 2024-05-09T20:16:51.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.940+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~ZDHPZBS3IHTH.0 size=1048576 2024-05-09T20:16:51.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.944+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~ZDHPZBS3IHTH.1 size=1048576 2024-05-09T20:16:51.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.948+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~ZDHPZBS3IHTH.2 size=637322 2024-05-09T20:16:51.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.952+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=ZDHPZBS3IHTH tag=P8Y46X9SD8BA type=write size=3783050 2024-05-09T20:16:51.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.956+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~LCABKM4LPTV5.0 size=1048576 2024-05-09T20:16:51.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.960+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~LCABKM4LPTV5.1 size=1048576 2024-05-09T20:16:51.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.964+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~LCABKM4LPTV5.2 size=149098 2024-05-09T20:16:51.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.964+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=LCABKM4LPTV5 tag=I6VUV68D8EPY type=write size=3294826 2024-05-09T20:16:51.958 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.964+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~NYP4HRPAOKCU.0 size=743015 2024-05-09T20:16:51.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.972+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=NYP4HRPAOKCU tag=1JBXXOFN0S3H type=write size=1791591 2024-05-09T20:16:51.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.972+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=OLSB3J9LYR1V type=del size=958423 2024-05-09T20:16:51.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.976+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~BGQ0A48LDSCK.0 size=1048576 2024-05-09T20:16:51.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.980+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~BGQ0A48LDSCK.1 size=1048576 2024-05-09T20:16:51.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.980+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~BGQ0A48LDSCK.2 size=1037772 2024-05-09T20:16:51.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.988+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=BGQ0A48LDSCK tag=WZVHAYYHEHG1 type=write size=4183500 2024-05-09T20:16:51.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.988+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=LJL4SY4SQDXW type=write size=703144 2024-05-09T20:16:51.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.992+0000 7fdc882e58c0 1 < canceled part key=_multipart_NBAECV9L85UK.2~HUOHEUA1ERDO.0 size=1048576 2024-05-09T20:16:51.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.992+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~HUOHEUA1ERDO.1 size=1048576 2024-05-09T20:16:51.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:51.996+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~HUOHEUA1ERDO.2 size=192857 2024-05-09T20:16:51.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.000+0000 7fdc882e58c0 1 < canceled multipart upload key=NBAECV9L85UK upload=HUOHEUA1ERDO tag=3ZLAZWLVX4IT type=write size=3338585 2024-05-09T20:16:51.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.004+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=5LDOYN1E1U0W type=del size=3253628 2024-05-09T20:16:52.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.008+0000 7fdc882e58c0 1 < suggested update operation key=O07UHAIJZF0R tag=ILQDSPN3I9CX type=write size=505627 2024-05-09T20:16:52.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.012+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~M5OGDVYASDXX.0 size=1048576 2024-05-09T20:16:52.008 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~M5OGDVYASDXX.1 size=94682 2024-05-09T20:16:52.008 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.016+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=M5OGDVYASDXX tag=JPU8ISCUYLY2 type=write size=2191834 2024-05-09T20:16:52.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.020+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=6K0QENK5PUP1 type=del 2024-05-09T20:16:52.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.020+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=AUIO6MTM0MMB type=del size=2713813 2024-05-09T20:16:52.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.028+0000 7fdc882e58c0 1 < suggested remove operation key=G0DX1F257RLD tag=WNNNYYZAP5UO type=del 2024-05-09T20:16:52.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.032+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=DKCND8Z6LDN3 type=del size=3415385 2024-05-09T20:16:52.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.032+0000 7fdc882e58c0 1 < canceled part key=_multipart_WC5YH0OU51WX.2~Q5VAP3WS7SI7.0 size=1048576 2024-05-09T20:16:52.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.032+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~Q5VAP3WS7SI7.1 size=195854 2024-05-09T20:16:52.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.040+0000 7fdc882e58c0 1 < canceled multipart upload key=WC5YH0OU51WX upload=Q5VAP3WS7SI7 tag=FB7Q9IC5100N type=write size=2293006 2024-05-09T20:16:52.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.044+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~HSCEI0DZT52A.0 size=999094 2024-05-09T20:16:52.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.044+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=HSCEI0DZT52A tag=8FSJH9IRG0EM type=write size=2047670 2024-05-09T20:16:52.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.052+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=ONG3FL6D0NS4 type=del size=941867 2024-05-09T20:16:52.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.056+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~6VUUZPQ5UZUA.0 size=1048576 2024-05-09T20:16:52.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.060+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~6VUUZPQ5UZUA.1 size=741240 2024-05-09T20:16:52.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.060+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=6VUUZPQ5UZUA tag=DMCN1E3G3Q8H type=write size=2838392 2024-05-09T20:16:52.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.060+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~5S5WA6FTKY5R.0 size=1048576 2024-05-09T20:16:52.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.064+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~5S5WA6FTKY5R.1 size=1048576 2024-05-09T20:16:52.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.068+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~5S5WA6FTKY5R.2 size=361103 2024-05-09T20:16:52.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.072+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=5S5WA6FTKY5R tag=FH3696QPS1DQ type=write size=3506831 2024-05-09T20:16:52.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.076+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~B4PSVH7SRWN1.0 size=879279 2024-05-09T20:16:52.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.076+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=B4PSVH7SRWN1 tag=15ZQ7GNE7IZU type=write size=1927855 2024-05-09T20:16:52.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.080+0000 7fdc882e58c0 1 < canceled part key=_multipart_G0DX1F257RLD.2~UCP9554R242F.0 size=1048576 2024-05-09T20:16:52.073 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.080+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~UCP9554R242F.1 size=737355 2024-05-09T20:16:52.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.088+0000 7fdc882e58c0 1 < canceled multipart upload key=G0DX1F257RLD upload=UCP9554R242F tag=3D9EO6Y7QCFN type=write size=2834507 2024-05-09T20:16:52.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.092+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=VG43AU5ERGO6 type=del size=328556 2024-05-09T20:16:52.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.096+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~JPNQS6XQAH2N.0 size=1048576 2024-05-09T20:16:52.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.096+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~JPNQS6XQAH2N.1 size=1048576 2024-05-09T20:16:52.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.100+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~JPNQS6XQAH2N.2 size=197719 2024-05-09T20:16:52.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.108+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=JPNQS6XQAH2N tag=9RC0Z90BURDC type=write size=3343447 2024-05-09T20:16:52.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.112+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~MH6ZEGNN0UO8.0 size=384891 2024-05-09T20:16:52.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.112+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=MH6ZEGNN0UO8 tag=NIP7S7112S0Q type=write size=1433467 2024-05-09T20:16:52.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.116+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=MUH64S6GNNMB type=del 2024-05-09T20:16:52.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.124+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~8ZNJHFJ86IAZ.0 size=1048576 2024-05-09T20:16:52.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.124+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~8ZNJHFJ86IAZ.1 size=1048576 2024-05-09T20:16:52.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.128+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~8ZNJHFJ86IAZ.2 size=420883 2024-05-09T20:16:52.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.128+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=8ZNJHFJ86IAZ tag=C3KT91PY05GM type=write size=3566611 2024-05-09T20:16:52.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.132+0000 7fdc882e58c0 1 < completed write operation key=346HV1XQKCOZ tag=BGNL7SCM0U4U type=write size=923087 2024-05-09T20:16:52.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.136+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~HC1GNGQGVF7D.0 size=881061 2024-05-09T20:16:52.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.136+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=HC1GNGQGVF7D tag=Y0JQDC20XIVN type=write size=1929637 2024-05-09T20:16:52.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.140+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~L8OYZAK9PPFZ.0 size=93225 2024-05-09T20:16:52.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.148+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=L8OYZAK9PPFZ tag=IBDDTGYNPOGL type=write size=1141801 2024-05-09T20:16:52.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.152+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~OQCOKPK58MJB.0 size=1048576 2024-05-09T20:16:52.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.156+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~OQCOKPK58MJB.1 size=1048576 2024-05-09T20:16:52.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.160+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~OQCOKPK58MJB.2 size=708349 2024-05-09T20:16:52.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.160+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=OQCOKPK58MJB tag=AW39M81NHW53 type=write size=3854077 2024-05-09T20:16:52.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.164+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=2ZNYAY5D7I9V type=del 2024-05-09T20:16:52.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.168+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~P6LJU8IEW4Z9.0 size=535937 2024-05-09T20:16:52.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.168+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=P6LJU8IEW4Z9 tag=8FLANN8GBFXN type=write size=1584513 2024-05-09T20:16:52.164 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.172+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~LCABKM4LPTV5.0 size=1048576 2024-05-09T20:16:52.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.176+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~LCABKM4LPTV5.1 size=1048576 2024-05-09T20:16:52.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.180+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~LCABKM4LPTV5.2 size=149098 2024-05-09T20:16:52.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.184+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=LCABKM4LPTV5 tag=I6VUV68D8EPY type=write size=3294826 2024-05-09T20:16:52.179 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.188+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=DPNKB7RFM0DV type=del size=2851023 2024-05-09T20:16:52.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.192+0000 7fdc882e58c0 1 < canceled operation key=5H2GWM9LQELZ tag=OLSB3J9LYR1V type=del size=958423 2024-05-09T20:16:52.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.196+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=K49GCT03T4UD type=write size=17305 2024-05-09T20:16:52.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.200+0000 7fdc882e58c0 1 < suggested remove operation key=HPK2T0629B1P tag=LJL4SY4SQDXW type=write 2024-05-09T20:16:52.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.204+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=ZFX0FHJI35BO type=del size=2791097 2024-05-09T20:16:52.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.208+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=5LDOYN1E1U0W type=del 2024-05-09T20:16:52.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.212+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=HBI83H35AO9S type=del size=1841260 2024-05-09T20:16:52.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.216+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~M5OGDVYASDXX.0 size=1048576 2024-05-09T20:16:52.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.220+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~M5OGDVYASDXX.1 size=94682 2024-05-09T20:16:52.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.224+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=M5OGDVYASDXX tag=JPU8ISCUYLY2 type=write size=2191834 2024-05-09T20:16:52.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.228+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=XZND9HP7VW1A type=del size=473630 2024-05-09T20:16:52.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.236+0000 7fdc882e58c0 1 < suggested update operation key=346HV1XQKCOZ tag=AUIO6MTM0MMB type=del size=2713813 2024-05-09T20:16:52.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.240+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=RBUL5QAJDNCO type=del size=2736788 2024-05-09T20:16:52.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.248+0000 7fdc882e58c0 1 < suggested update operation key=NNGWUSCETMH4 tag=DKCND8Z6LDN3 type=del size=3415385 2024-05-09T20:16:52.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.252+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=1RWGGYTB3PY6 type=del size=2090347 2024-05-09T20:16:52.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.256+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~HSCEI0DZT52A.0 size=999094 2024-05-09T20:16:52.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.260+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=HSCEI0DZT52A tag=8FSJH9IRG0EM type=write size=2047670 2024-05-09T20:16:52.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.268+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~U55LI0REVQ8Y.0 size=1048576 2024-05-09T20:16:52.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.272+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~U55LI0REVQ8Y.1 size=651361 2024-05-09T20:16:52.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.272+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=U55LI0REVQ8Y tag=4CSG58IT06ZG type=write size=2748513 2024-05-09T20:16:52.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.272+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~6VUUZPQ5UZUA.0 size=1048576 2024-05-09T20:16:52.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.276+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~6VUUZPQ5UZUA.1 size=741240 2024-05-09T20:16:52.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.280+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=6VUUZPQ5UZUA tag=DMCN1E3G3Q8H type=write size=2838392 2024-05-09T20:16:52.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.284+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=HG6UGBKOZUW3 type=del size=3273800 2024-05-09T20:16:52.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.288+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~B4PSVH7SRWN1.0 size=879279 2024-05-09T20:16:52.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.296+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=B4PSVH7SRWN1 tag=15ZQ7GNE7IZU type=write size=1927855 2024-05-09T20:16:52.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.300+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~A8WQ7GCLFZUA.0 size=1048576 2024-05-09T20:16:52.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.304+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~A8WQ7GCLFZUA.1 size=1048576 2024-05-09T20:16:52.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.308+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~A8WQ7GCLFZUA.2 size=418253 2024-05-09T20:16:52.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.308+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=A8WQ7GCLFZUA tag=NUTHAQE4PIAK type=write size=3563981 2024-05-09T20:16:52.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.312+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=VG43AU5ERGO6 type=del 2024-05-09T20:16:52.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.316+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=1FLCKVI498MC type=del size=2172387 2024-05-09T20:16:52.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.320+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~MH6ZEGNN0UO8.0 size=384891 2024-05-09T20:16:52.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.324+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=MH6ZEGNN0UO8 tag=NIP7S7112S0Q type=write size=1433467 2024-05-09T20:16:52.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.328+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=S370XQAJ7WD9 type=del size=3118617 2024-05-09T20:16:52.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.332+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~8ZNJHFJ86IAZ.0 size=1048576 2024-05-09T20:16:52.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.336+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~8ZNJHFJ86IAZ.1 size=1048576 2024-05-09T20:16:52.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.336+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~8ZNJHFJ86IAZ.2 size=420883 2024-05-09T20:16:52.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.340+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=8ZNJHFJ86IAZ tag=C3KT91PY05GM type=write size=3566611 2024-05-09T20:16:52.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.348+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~LOOCL44KP663.0 size=1002721 2024-05-09T20:16:52.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.348+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=LOOCL44KP663 tag=O4XOJ1SEUOZ2 type=write size=2051297 2024-05-09T20:16:52.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.352+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~HC1GNGQGVF7D.0 size=881061 2024-05-09T20:16:52.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.356+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=HC1GNGQGVF7D tag=Y0JQDC20XIVN type=write size=1929637 2024-05-09T20:16:52.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.364+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~GCMNOL7VA42J.0 size=509686 2024-05-09T20:16:52.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.364+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=GCMNOL7VA42J tag=6GT8QS81R4LL type=write size=1558262 2024-05-09T20:16:52.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.368+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~OQCOKPK58MJB.0 size=1048576 2024-05-09T20:16:52.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.368+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~OQCOKPK58MJB.1 size=1048576 2024-05-09T20:16:52.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.372+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~OQCOKPK58MJB.2 size=708349 2024-05-09T20:16:52.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.376+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=OQCOKPK58MJB tag=AW39M81NHW53 type=write size=3854077 2024-05-09T20:16:52.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.380+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=91WTAGLEEKIS type=write size=97613 2024-05-09T20:16:52.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.384+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~P6LJU8IEW4Z9.0 size=535937 2024-05-09T20:16:52.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.392+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=P6LJU8IEW4Z9 tag=8FLANN8GBFXN type=write size=1584513 2024-05-09T20:16:52.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.396+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~9E0YUME2Q2AA.0 size=34048 2024-05-09T20:16:52.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.396+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=9E0YUME2Q2AA tag=JPAHUL6278BK type=write size=1082624 2024-05-09T20:16:52.392 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.400+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=DPNKB7RFM0DV type=del 2024-05-09T20:16:52.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.404+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=S3IWH0D93BCV type=write size=378521 2024-05-09T20:16:52.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.408+0000 7fdc882e58c0 -1 < failed to complete operation key=HPK2T0629B1P tag=K49GCT03T4UD type=write size=17305: (22) Invalid argument 2024-05-09T20:16:52.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.412+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=W11N7JYC8Q88 type=del size=1439155 2024-05-09T20:16:52.406 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.416+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=ZFX0FHJI35BO type=del 2024-05-09T20:16:52.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.416+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=0HZGJIZEQ7WS type=write size=185288 2024-05-09T20:16:52.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.420+0000 7fdc882e58c0 1 < canceled operation key=NBAECV9L85UK tag=HBI83H35AO9S type=del size=1841260 2024-05-09T20:16:52.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.424+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=7B9JABGOYYNJ type=del size=3514961 2024-05-09T20:16:52.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.432+0000 7fdc882e58c0 1 < suggested remove operation key=FNB9B18LIA8L tag=XZND9HP7VW1A type=del 2024-05-09T20:16:52.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.440+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~QBM536JTCYS3.0 size=1048576 2024-05-09T20:16:52.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.440+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~QBM536JTCYS3.1 size=1048576 2024-05-09T20:16:52.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.444+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~QBM536JTCYS3.2 size=548471 2024-05-09T20:16:52.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.444+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=QBM536JTCYS3 tag=XAGLNX9OI90S type=write size=3694199 2024-05-09T20:16:52.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.452+0000 7fdc882e58c0 1 < suggested update operation key=NYHR6GTPQYZS tag=RBUL5QAJDNCO type=del size=2736788 2024-05-09T20:16:52.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.456+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=6QLWU2OMS1ZA type=del size=3093451 2024-05-09T20:16:52.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.460+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=1RWGGYTB3PY6 type=del 2024-05-09T20:16:52.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.468+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~E1YFTRKB6K5V.0 size=1048576 2024-05-09T20:16:52.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.472+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~E1YFTRKB6K5V.1 size=1048576 2024-05-09T20:16:52.465 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.472+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~E1YFTRKB6K5V.2 size=981928 2024-05-09T20:16:52.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.472+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=E1YFTRKB6K5V tag=2N4IPKCQP8WG type=write size=4127656 2024-05-09T20:16:52.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.476+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~U55LI0REVQ8Y.0 size=1048576 2024-05-09T20:16:52.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.480+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~U55LI0REVQ8Y.1 size=651361 2024-05-09T20:16:52.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.488+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=U55LI0REVQ8Y tag=4CSG58IT06ZG type=write size=2748513 2024-05-09T20:16:52.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.488+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=8W2ZUK45GOEE type=write size=637078 2024-05-09T20:16:52.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.492+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=HG6UGBKOZUW3 type=del 2024-05-09T20:16:52.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.500+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~EA217V8C98IZ.0 size=528060 2024-05-09T20:16:52.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.500+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=EA217V8C98IZ tag=EMM14NKWALVR type=write size=1576636 2024-05-09T20:16:52.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.504+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~A8WQ7GCLFZUA.0 size=1048576 2024-05-09T20:16:52.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.504+0000 7fdc882e58c0 1 < canceled part key=_multipart_FNB9B18LIA8L.2~A8WQ7GCLFZUA.1 size=1048576 2024-05-09T20:16:52.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.508+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~A8WQ7GCLFZUA.2 size=418253 2024-05-09T20:16:52.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.512+0000 7fdc882e58c0 1 < canceled multipart upload key=FNB9B18LIA8L upload=A8WQ7GCLFZUA tag=NUTHAQE4PIAK type=write size=3563981 2024-05-09T20:16:52.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.516+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=VZHNO36VZ1Y8 type=del size=4028619 2024-05-09T20:16:52.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.520+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=1FLCKVI498MC type=del 2024-05-09T20:16:52.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.524+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=W3G63NEOK5RL type=del size=4169323 2024-05-09T20:16:52.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.528+0000 7fdc882e58c0 1 < suggested remove operation key=QM53IENH9CX4 tag=S370XQAJ7WD9 type=del 2024-05-09T20:16:52.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.532+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=SIG765TSSPKY type=del size=1876498 2024-05-09T20:16:52.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.536+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~LOOCL44KP663.0 size=1002721 2024-05-09T20:16:52.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.540+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=LOOCL44KP663 tag=O4XOJ1SEUOZ2 type=write size=2051297 2024-05-09T20:16:52.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.544+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=26WB35FFGNNQ type=del size=543512 2024-05-09T20:16:52.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.548+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~GCMNOL7VA42J.0 size=509686 2024-05-09T20:16:52.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.556+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=GCMNOL7VA42J tag=6GT8QS81R4LL type=write size=1558262 2024-05-09T20:16:52.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.560+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~MKWGIB18LJNC.0 size=357866 2024-05-09T20:16:52.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.560+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=MKWGIB18LJNC tag=LXL27ZH2ZN01 type=write size=1406442 2024-05-09T20:16:52.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.564+0000 7fdc882e58c0 1 < completed write operation key=5NEOR8OOFAFZ tag=91WTAGLEEKIS type=write size=97613 2024-05-09T20:16:52.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.568+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=L6I1422ZBXFN type=del size=3925654 2024-05-09T20:16:52.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.572+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~9E0YUME2Q2AA.0 size=34048 2024-05-09T20:16:52.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.580+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=9E0YUME2Q2AA tag=JPAHUL6278BK type=write size=1082624 2024-05-09T20:16:52.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.584+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~4FNANNWX4V6G.0 size=1048576 2024-05-09T20:16:52.579 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~4FNANNWX4V6G.1 size=24415 2024-05-09T20:16:52.579 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.588+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=4FNANNWX4V6G tag=M2T8D2ERNF19 type=write size=2121567 2024-05-09T20:16:52.583 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.592+0000 7fdc882e58c0 1 < completed write operation key=VZ7XOAT899GU tag=S3IWH0D93BCV type=write size=378521 2024-05-09T20:16:52.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.596+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=0GP2O3YU6702 type=del size=1705405 2024-05-09T20:16:52.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.600+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=W11N7JYC8Q88 type=del 2024-05-09T20:16:52.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.600+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=R5C7G9561J7E type=write size=200256 2024-05-09T20:16:52.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.604+0000 7fdc882e58c0 1 < completed write operation key=5NEOR8OOFAFZ tag=0HZGJIZEQ7WS type=write size=185288 2024-05-09T20:16:52.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.608+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=KA7ASGNHK6CJ type=del size=2343680 2024-05-09T20:16:52.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.612+0000 7fdc882e58c0 1 < canceled operation key=CNV7NHP27JZ7 tag=7B9JABGOYYNJ type=del size=3514961 2024-05-09T20:16:52.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.616+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=6UQSKC3INPD7 type=del size=1053985 2024-05-09T20:16:52.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.620+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~QBM536JTCYS3.0 size=1048576 2024-05-09T20:16:52.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.620+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~QBM536JTCYS3.1 size=1048576 2024-05-09T20:16:52.616 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.624+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~QBM536JTCYS3.2 size=548471 2024-05-09T20:16:52.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.632+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=QBM536JTCYS3 tag=XAGLNX9OI90S type=write size=3694199 2024-05-09T20:16:52.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.636+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~LCB61SG3JM0U.0 size=1008287 2024-05-09T20:16:52.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.636+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=LCB61SG3JM0U tag=EF5KPIFGI7B5 type=write size=2056863 2024-05-09T20:16:52.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.640+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=6QLWU2OMS1ZA type=del 2024-05-09T20:16:52.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.644+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=L54KXK5MY8CB type=del size=3481268 2024-05-09T20:16:52.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.648+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~E1YFTRKB6K5V.0 size=1048576 2024-05-09T20:16:52.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.648+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~E1YFTRKB6K5V.1 size=1048576 2024-05-09T20:16:52.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.652+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~E1YFTRKB6K5V.2 size=981928 2024-05-09T20:16:52.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.656+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=E1YFTRKB6K5V tag=2N4IPKCQP8WG type=write size=4127656 2024-05-09T20:16:52.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.660+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=4G5BUMTL3UBH type=del size=2017940 2024-05-09T20:16:52.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.668+0000 7fdc882e58c0 1 < suggested remove operation key=346HV1XQKCOZ tag=8W2ZUK45GOEE type=write 2024-05-09T20:16:52.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.672+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=BRODONCNZ8KR type=write size=251727 2024-05-09T20:16:52.667 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.676+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~EA217V8C98IZ.0 size=528060 2024-05-09T20:16:52.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.680+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=EA217V8C98IZ tag=EMM14NKWALVR type=write size=1576636 2024-05-09T20:16:52.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.684+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=CTW6S4V6DNU2 type=del size=1434485 2024-05-09T20:16:52.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.688+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=VZHNO36VZ1Y8 type=del 2024-05-09T20:16:52.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.692+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=ZCQIYTNTJS36 type=del size=3161776 2024-05-09T20:16:52.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.696+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=W3G63NEOK5RL type=del 2024-05-09T20:16:52.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.700+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=A75IK0H3PXRD type=del size=489175 2024-05-09T20:16:52.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.704+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=SIG765TSSPKY type=del 2024-05-09T20:16:52.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.708+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~L9O8BI48MHNA.0 size=1048576 2024-05-09T20:16:52.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~L9O8BI48MHNA.1 size=384882 2024-05-09T20:16:52.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.712+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=L9O8BI48MHNA tag=XHOM55CQ4DNJ type=write size=2482034 2024-05-09T20:16:52.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.716+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=26WB35FFGNNQ type=del 2024-05-09T20:16:52.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.720+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=YVCUPN2AMOA6 type=write size=195418 2024-05-09T20:16:52.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.724+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~MKWGIB18LJNC.0 size=357866 2024-05-09T20:16:52.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.728+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=MKWGIB18LJNC tag=LXL27ZH2ZN01 type=write size=1406442 2024-05-09T20:16:52.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~XMAM3FZPMSM3.0 size=1048576 2024-05-09T20:16:52.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~XMAM3FZPMSM3.1 size=345753 2024-05-09T20:16:52.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.736+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=XMAM3FZPMSM3 tag=AKJJXZ5KTGNR type=write size=2442905 2024-05-09T20:16:52.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.740+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=L6I1422ZBXFN type=del 2024-05-09T20:16:52.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.744+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=FONTMHRD5HUS type=del size=339260 2024-05-09T20:16:52.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.748+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~4FNANNWX4V6G.0 size=1048576 2024-05-09T20:16:52.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.748+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~4FNANNWX4V6G.1 size=24415 2024-05-09T20:16:52.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.756+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=4FNANNWX4V6G tag=M2T8D2ERNF19 type=write size=2121567 2024-05-09T20:16:52.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.756+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=NK0EYHK8OBRN type=del size=1740025 2024-05-09T20:16:52.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.764+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=0GP2O3YU6702 type=del 2024-05-09T20:16:52.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.768+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~MI0BDY4WRZ1A.0 size=486075 2024-05-09T20:16:52.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.768+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=MI0BDY4WRZ1A tag=W3EV196F1QLX type=write size=1534651 2024-05-09T20:16:52.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.772+0000 7fdc882e58c0 1 < completed write operation key=EK78CNKUHNKO tag=R5C7G9561J7E type=write size=200256 2024-05-09T20:16:52.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.776+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=ZWB1NL4P2YZC type=del size=1686246 2024-05-09T20:16:52.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.780+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=KA7ASGNHK6CJ type=del 2024-05-09T20:16:52.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.780+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=GHEH8J6MHB08 type=write size=848755 2024-05-09T20:16:52.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.784+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=6UQSKC3INPD7 type=del 2024-05-09T20:16:52.782 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.792+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~EG7HQVITVMQW.0 size=431791 2024-05-09T20:16:52.782 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.792+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=EG7HQVITVMQW tag=UW12CV25FHHB type=write size=1480367 2024-05-09T20:16:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.792+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~LCB61SG3JM0U.0 size=1008287 2024-05-09T20:16:52.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.796+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=LCB61SG3JM0U tag=EF5KPIFGI7B5 type=write size=2056863 2024-05-09T20:16:52.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.800+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=8F6WETZO1LCN type=del size=2275863 2024-05-09T20:16:52.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.804+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=L54KXK5MY8CB type=del 2024-05-09T20:16:52.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.808+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=1IQMULDPUTOQ type=write size=969859 2024-05-09T20:16:52.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.812+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=4G5BUMTL3UBH type=del 2024-05-09T20:16:52.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.816+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~O0M1BT96CVG5.0 size=1048576 2024-05-09T20:16:52.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.820+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~O0M1BT96CVG5.1 size=68973 2024-05-09T20:16:52.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.820+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=O0M1BT96CVG5 tag=AI0OQZADHI7M type=write size=2166125 2024-05-09T20:16:52.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.824+0000 7fdc882e58c0 1 < completed write operation key=T6T3IOW9VPA0 tag=BRODONCNZ8KR type=write size=251727 2024-05-09T20:16:52.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.828+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=RF62A5FG4MOA type=del size=3715392 2024-05-09T20:16:52.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.832+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=CTW6S4V6DNU2 type=del 2024-05-09T20:16:52.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.832+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=61DFVF1PO8LR type=write size=161542 2024-05-09T20:16:52.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.836+0000 7fdc882e58c0 1 < canceled operation key=346HV1XQKCOZ tag=ZCQIYTNTJS36 type=del size=3161776 2024-05-09T20:16:52.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.844+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~YZS4GIR7KXH3.0 size=873775 2024-05-09T20:16:52.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.844+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=YZS4GIR7KXH3 tag=73957W30S5FY type=write size=1922351 2024-05-09T20:16:52.839 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.848+0000 7fdc882e58c0 1 < canceled operation key=P2DNA4DGI54P tag=A75IK0H3PXRD type=del size=489175 2024-05-09T20:16:52.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.852+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~9GUA98PM9PM0.0 size=1048576 2024-05-09T20:16:52.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.856+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~9GUA98PM9PM0.1 size=1048576 2024-05-09T20:16:52.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.856+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~9GUA98PM9PM0.2 size=274580 2024-05-09T20:16:52.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.856+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=9GUA98PM9PM0 tag=ORK2GPQ4927A type=write size=3420308 2024-05-09T20:16:52.853 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.860+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~L9O8BI48MHNA.0 size=1048576 2024-05-09T20:16:52.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.864+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~L9O8BI48MHNA.1 size=384882 2024-05-09T20:16:52.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.868+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=L9O8BI48MHNA tag=XHOM55CQ4DNJ type=write size=2482034 2024-05-09T20:16:52.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.872+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=1RS61NEVPWKG type=del size=3798354 2024-05-09T20:16:52.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.876+0000 7fdc882e58c0 1 < completed write operation key=T6T3IOW9VPA0 tag=YVCUPN2AMOA6 type=write size=195418 2024-05-09T20:16:52.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.876+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=DQ9FKJ10CWO0 type=del size=3096979 2024-05-09T20:16:52.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.880+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~XMAM3FZPMSM3.0 size=1048576 2024-05-09T20:16:52.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.880+0000 7fdc882e58c0 1 < canceled part key=_multipart_NBAECV9L85UK.2~XMAM3FZPMSM3.1 size=345753 2024-05-09T20:16:52.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.888+0000 7fdc882e58c0 1 < canceled multipart upload key=NBAECV9L85UK upload=XMAM3FZPMSM3 tag=AKJJXZ5KTGNR type=write size=2442905 2024-05-09T20:16:52.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.896+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~Y8RE7U1NTZJ1.0 size=1048576 2024-05-09T20:16:52.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.896+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~Y8RE7U1NTZJ1.1 size=1048576 2024-05-09T20:16:52.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~Y8RE7U1NTZJ1.2 size=32068 2024-05-09T20:16:52.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.900+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=Y8RE7U1NTZJ1 tag=FTVNSTFDN2IQ type=write size=3177796 2024-05-09T20:16:52.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.904+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=FONTMHRD5HUS type=del 2024-05-09T20:16:52.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.908+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=EO0Z3LG52ZO4 type=del size=2116486 2024-05-09T20:16:52.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.908+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=NK0EYHK8OBRN type=del 2024-05-09T20:16:52.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.912+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=LJ2LFMLPFLJT type=del size=3986137 2024-05-09T20:16:52.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.916+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~MI0BDY4WRZ1A.0 size=486075 2024-05-09T20:16:52.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.920+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=MI0BDY4WRZ1A tag=W3EV196F1QLX type=write size=1534651 2024-05-09T20:16:52.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.924+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=6Z1Z7ECPZQGZ type=write size=232081 2024-05-09T20:16:52.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.932+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=ZWB1NL4P2YZC type=del size=1686246 2024-05-09T20:16:52.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.936+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=686YM4Z7HOJB type=del size=2960706 2024-05-09T20:16:52.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.936+0000 7fdc882e58c0 1 < completed write operation key=VZ7XOAT899GU tag=GHEH8J6MHB08 type=write size=848755 2024-05-09T20:16:52.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.940+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=7C5F2FZG2NAL type=write size=144867 2024-05-09T20:16:52.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.944+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~EG7HQVITVMQW.0 size=431791 2024-05-09T20:16:52.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.948+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=EG7HQVITVMQW tag=UW12CV25FHHB type=write size=1480367 2024-05-09T20:16:52.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.956+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~T7GF31L2GC7E.0 size=652591 2024-05-09T20:16:52.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.956+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=T7GF31L2GC7E tag=BR9PPL1AUJQ3 type=write size=1701167 2024-05-09T20:16:52.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.960+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=8F6WETZO1LCN type=del 2024-05-09T20:16:52.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.964+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~6VU8XYY0ZBB1.0 size=899763 2024-05-09T20:16:52.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.964+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=6VU8XYY0ZBB1 tag=00VV67BOHT72 type=write size=1948339 2024-05-09T20:16:52.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.968+0000 7fdc882e58c0 1 < completed write operation key=NNGWUSCETMH4 tag=1IQMULDPUTOQ type=write size=969859 2024-05-09T20:16:52.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.972+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=F64GHDU2M7WS type=del size=4013829 2024-05-09T20:16:52.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.972+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~O0M1BT96CVG5.0 size=1048576 2024-05-09T20:16:52.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.976+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~O0M1BT96CVG5.1 size=68973 2024-05-09T20:16:52.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.980+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=O0M1BT96CVG5 tag=AI0OQZADHI7M type=write size=2166125 2024-05-09T20:16:52.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.984+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=5B86QVIATPCZ type=del size=1814512 2024-05-09T20:16:52.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.988+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=RF62A5FG4MOA type=del 2024-05-09T20:16:52.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.992+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~9KOM6LBVH7QU.0 size=1048576 2024-05-09T20:16:52.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:52.996+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~9KOM6LBVH7QU.1 size=1048576 2024-05-09T20:16:52.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.000+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~9KOM6LBVH7QU.2 size=380078 2024-05-09T20:16:52.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.000+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=9KOM6LBVH7QU tag=XEOBI0D6TWC4 type=write size=3525806 2024-05-09T20:16:52.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.000+0000 7fdc882e58c0 1 < completed write operation key=J2Q86HKRMAKE tag=61DFVF1PO8LR type=write size=161542 2024-05-09T20:16:52.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.004+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=HTXSNXIOLOA3 type=del size=1675796 2024-05-09T20:16:53.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.008+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~YZS4GIR7KXH3.0 size=873775 2024-05-09T20:16:53.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.012+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=YZS4GIR7KXH3 tag=73957W30S5FY type=write size=1922351 2024-05-09T20:16:53.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.020+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~TI5GT7WZY4NC.0 size=1048576 2024-05-09T20:16:53.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.020+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~TI5GT7WZY4NC.1 size=247220 2024-05-09T20:16:53.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.020+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=TI5GT7WZY4NC tag=ZD6Y4YPLQ2P1 type=write size=2344372 2024-05-09T20:16:53.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.024+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~9GUA98PM9PM0.0 size=1048576 2024-05-09T20:16:53.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.028+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~9GUA98PM9PM0.1 size=1048576 2024-05-09T20:16:53.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.028+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~9GUA98PM9PM0.2 size=274580 2024-05-09T20:16:53.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.036+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=9GUA98PM9PM0 tag=ORK2GPQ4927A type=write size=3420308 2024-05-09T20:16:53.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.040+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=6SKXIFJGR2QR type=del size=703206 2024-05-09T20:16:53.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.044+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=1RS61NEVPWKG type=del 2024-05-09T20:16:53.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.048+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=JCOXCWK09H33 type=del size=3857211 2024-05-09T20:16:53.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.052+0000 7fdc882e58c0 1 < canceled operation key=AJANS5ADJO5B tag=DQ9FKJ10CWO0 type=del size=3096979 2024-05-09T20:16:53.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.056+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=05MOCYU00CGL type=del size=190130 2024-05-09T20:16:53.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.060+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~Y8RE7U1NTZJ1.0 size=1048576 2024-05-09T20:16:53.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.060+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~Y8RE7U1NTZJ1.1 size=1048576 2024-05-09T20:16:53.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.064+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~Y8RE7U1NTZJ1.2 size=32068 2024-05-09T20:16:53.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.072+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=Y8RE7U1NTZJ1 tag=FTVNSTFDN2IQ type=write size=3177796 2024-05-09T20:16:53.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.076+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~6DA2AOPGNRAW.0 size=1048576 2024-05-09T20:16:53.073 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.080+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~6DA2AOPGNRAW.1 size=1048576 2024-05-09T20:16:53.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.084+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~6DA2AOPGNRAW.2 size=628107 2024-05-09T20:16:53.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.084+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=6DA2AOPGNRAW tag=6XL03W54ZDPP type=write size=3773835 2024-05-09T20:16:53.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.088+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=EO0Z3LG52ZO4 type=del 2024-05-09T20:16:53.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.092+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~9JRRAUDFBCYV.0 size=468456 2024-05-09T20:16:53.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.092+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=9JRRAUDFBCYV tag=0C7P10YATPW8 type=write size=1517032 2024-05-09T20:16:53.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.096+0000 7fdc882e58c0 1 < canceled operation key=T6T3IOW9VPA0 tag=LJ2LFMLPFLJT type=del size=3986137 2024-05-09T20:16:53.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.100+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=IML9E0MU1SRX type=del size=3762060 2024-05-09T20:16:53.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.104+0000 7fdc882e58c0 1 < completed write operation key=8FFZZDURUWM1 tag=6Z1Z7ECPZQGZ type=write size=232081 2024-05-09T20:16:53.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.108+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=0IIF2B4JOY59 type=del size=1732096 2024-05-09T20:16:53.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.116+0000 7fdc882e58c0 1 < suggested update operation key=AJANS5ADJO5B tag=686YM4Z7HOJB type=del size=2960706 2024-05-09T20:16:53.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.120+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~V8BBNJYOS1JT.0 size=1048576 2024-05-09T20:16:53.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.124+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~V8BBNJYOS1JT.1 size=686061 2024-05-09T20:16:53.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.124+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=V8BBNJYOS1JT tag=HQ19N7BFXFIN type=write size=2783213 2024-05-09T20:16:53.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.124+0000 7fdc882e58c0 1 < completed write operation key=5H2GWM9LQELZ tag=7C5F2FZG2NAL type=write size=144867 2024-05-09T20:16:53.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.128+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=RYW0O0ADYVG4 type=del size=3465626 2024-05-09T20:16:53.124 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.132+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~T7GF31L2GC7E.0 size=652591 2024-05-09T20:16:53.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.136+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=T7GF31L2GC7E tag=BR9PPL1AUJQ3 type=write size=1701167 2024-05-09T20:16:53.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.140+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=T844PK9G9R2N type=del size=4076807 2024-05-09T20:16:53.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.144+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~6VU8XYY0ZBB1.0 size=899763 2024-05-09T20:16:53.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.148+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=6VU8XYY0ZBB1 tag=00VV67BOHT72 type=write size=1948339 2024-05-09T20:16:53.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.152+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=NAUTPTLESZE1 type=del size=1229519 2024-05-09T20:16:53.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.156+0000 7fdc882e58c0 1 < suggested update operation key=HPK2T0629B1P tag=F64GHDU2M7WS type=del size=4013829 2024-05-09T20:16:53.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.160+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=H9IDM5VHPTI3 type=del size=835501 2024-05-09T20:16:53.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.160+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=5B86QVIATPCZ type=del 2024-05-09T20:16:53.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.164+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=BNL2TQYK3AM7 type=write size=392136 2024-05-09T20:16:53.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.168+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~9KOM6LBVH7QU.0 size=1048576 2024-05-09T20:16:53.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.168+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~9KOM6LBVH7QU.1 size=1048576 2024-05-09T20:16:53.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.172+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~9KOM6LBVH7QU.2 size=380078 2024-05-09T20:16:53.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.176+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=9KOM6LBVH7QU tag=XEOBI0D6TWC4 type=write size=3525806 2024-05-09T20:16:53.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.180+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~4DVIGA3J0NGJ.0 size=427869 2024-05-09T20:16:53.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.180+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=4DVIGA3J0NGJ tag=XBH40QTF5O4F type=write size=1476445 2024-05-09T20:16:53.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.184+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=HTXSNXIOLOA3 type=del 2024-05-09T20:16:53.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.188+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=5EBWPQM1PVGU type=del size=1361751 2024-05-09T20:16:53.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.188+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~TI5GT7WZY4NC.0 size=1048576 2024-05-09T20:16:53.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.192+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~TI5GT7WZY4NC.1 size=247220 2024-05-09T20:16:53.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.196+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=TI5GT7WZY4NC tag=ZD6Y4YPLQ2P1 type=write size=2344372 2024-05-09T20:16:53.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.200+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~0FPLAXR4E8M8.0 size=1048576 2024-05-09T20:16:53.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.204+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~0FPLAXR4E8M8.1 size=438767 2024-05-09T20:16:53.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.204+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=0FPLAXR4E8M8 tag=DC6R4ZPN1F3Z type=write size=2535919 2024-05-09T20:16:53.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.208+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=6SKXIFJGR2QR type=del 2024-05-09T20:16:53.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.208+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=9EGCVXZ14WXD type=del size=2981239 2024-05-09T20:16:53.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.212+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=JCOXCWK09H33 type=del 2024-05-09T20:16:53.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.216+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~2HYAY20O4MAK.0 size=1048576 2024-05-09T20:16:53.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.220+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~2HYAY20O4MAK.1 size=1048576 2024-05-09T20:16:53.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.220+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~2HYAY20O4MAK.2 size=275619 2024-05-09T20:16:53.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.220+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=2HYAY20O4MAK tag=PIKRMK0OP584 type=write size=3421347 2024-05-09T20:16:53.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.224+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=05MOCYU00CGL type=del 2024-05-09T20:16:53.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.228+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=69L4O20XXRYQ type=del size=4045348 2024-05-09T20:16:53.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.228+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~6DA2AOPGNRAW.0 size=1048576 2024-05-09T20:16:53.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.232+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~6DA2AOPGNRAW.1 size=1048576 2024-05-09T20:16:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.236+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~6DA2AOPGNRAW.2 size=628107 2024-05-09T20:16:53.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.240+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=6DA2AOPGNRAW tag=6XL03W54ZDPP type=write size=3773835 2024-05-09T20:16:53.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.240+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=LCU97RUC01GR type=write size=764723 2024-05-09T20:16:53.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.244+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~9JRRAUDFBCYV.0 size=468456 2024-05-09T20:16:53.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.248+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=9JRRAUDFBCYV tag=0C7P10YATPW8 type=write size=1517032 2024-05-09T20:16:53.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.256+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~XCTOC7B754B1.0 size=192732 2024-05-09T20:16:53.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.256+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=XCTOC7B754B1 tag=PBFNECEMOUMT type=write size=1241308 2024-05-09T20:16:53.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.256+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=IML9E0MU1SRX type=del 2024-05-09T20:16:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.264+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~VUKR41TE4ZQ2.0 size=628677 2024-05-09T20:16:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.264+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=VUKR41TE4ZQ2 tag=U4O3MD3OJ78O type=write size=1677253 2024-05-09T20:16:53.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.268+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=0IIF2B4JOY59 type=del 2024-05-09T20:16:53.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.272+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~WHHG9NF66KM3.0 size=1048576 2024-05-09T20:16:53.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.276+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~WHHG9NF66KM3.1 size=252862 2024-05-09T20:16:53.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.276+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=WHHG9NF66KM3 tag=C3WI8JCUDKD2 type=write size=2350014 2024-05-09T20:16:53.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.280+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~V8BBNJYOS1JT.0 size=1048576 2024-05-09T20:16:53.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.280+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~V8BBNJYOS1JT.1 size=686061 2024-05-09T20:16:53.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.288+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=V8BBNJYOS1JT tag=HQ19N7BFXFIN type=write size=2783213 2024-05-09T20:16:53.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.292+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~TQUOS8R2Y7PV.0 size=1048576 2024-05-09T20:16:53.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.296+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~TQUOS8R2Y7PV.1 size=507410 2024-05-09T20:16:53.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.296+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=TQUOS8R2Y7PV tag=DT5D73L8HFRD type=write size=2604562 2024-05-09T20:16:53.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.300+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=RYW0O0ADYVG4 type=del 2024-05-09T20:16:53.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.304+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~8793WPQ030PV.0 size=1048576 2024-05-09T20:16:53.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.308+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~8793WPQ030PV.1 size=1048576 2024-05-09T20:16:53.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.308+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~8793WPQ030PV.2 size=150010 2024-05-09T20:16:53.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.308+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=8793WPQ030PV tag=45S3VENOQ24K type=write size=3295738 2024-05-09T20:16:53.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.312+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=T844PK9G9R2N type=del 2024-05-09T20:16:53.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.320+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~CYUTTCQVTFB7.0 size=1048576 2024-05-09T20:16:53.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.320+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~CYUTTCQVTFB7.1 size=153466 2024-05-09T20:16:53.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.320+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=CYUTTCQVTFB7 tag=YJAM0X5E5KJL type=write size=2250618 2024-05-09T20:16:53.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.324+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=NAUTPTLESZE1 type=del 2024-05-09T20:16:53.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.324+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=RHA1EFE0177U type=del size=3171743 2024-05-09T20:16:53.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.328+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=H9IDM5VHPTI3 type=del 2024-05-09T20:16:53.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.332+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=QNYACYQ4PWPI type=write size=22668 2024-05-09T20:16:53.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.332+0000 7fdc882e58c0 1 < completed write operation key=VZ7XOAT899GU tag=BNL2TQYK3AM7 type=write size=392136 2024-05-09T20:16:53.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.336+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=WOSZ1FTFS58L type=del size=1060345 2024-05-09T20:16:53.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.340+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~4DVIGA3J0NGJ.0 size=427869 2024-05-09T20:16:53.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.344+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=4DVIGA3J0NGJ tag=XBH40QTF5O4F type=write size=1476445 2024-05-09T20:16:53.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.348+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~L3VZLBY2JCZD.0 size=1048576 2024-05-09T20:16:53.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.352+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~L3VZLBY2JCZD.1 size=751842 2024-05-09T20:16:53.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.352+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=L3VZLBY2JCZD tag=KPRZ2LND7G1P type=write size=2848994 2024-05-09T20:16:53.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.352+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=5EBWPQM1PVGU type=del 2024-05-09T20:16:53.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.360+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~K7IHOHP3Y6CX.0 size=85105 2024-05-09T20:16:53.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.360+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=K7IHOHP3Y6CX tag=VCUY3GEMD5FG type=write size=1133681 2024-05-09T20:16:53.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.360+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~0FPLAXR4E8M8.0 size=1048576 2024-05-09T20:16:53.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.364+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~0FPLAXR4E8M8.1 size=438767 2024-05-09T20:16:53.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.368+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=0FPLAXR4E8M8 tag=DC6R4ZPN1F3Z type=write size=2535919 2024-05-09T20:16:53.366 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.376+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~WLKFGHUWN91H.0 size=643148 2024-05-09T20:16:53.367 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.376+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=WLKFGHUWN91H tag=KX0S8D0KOCZ1 type=write size=1691724 2024-05-09T20:16:53.370 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.380+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=9EGCVXZ14WXD type=del 2024-05-09T20:16:53.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.380+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=UDSY9L1EN5QX type=del size=337950 2024-05-09T20:16:53.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.384+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~2HYAY20O4MAK.0 size=1048576 2024-05-09T20:16:53.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.384+0000 7fdc882e58c0 1 < canceled part key=_multipart_SNPNT5JFJOUS.2~2HYAY20O4MAK.1 size=1048576 2024-05-09T20:16:53.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.388+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~2HYAY20O4MAK.2 size=275619 2024-05-09T20:16:53.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.392+0000 7fdc882e58c0 1 < canceled multipart upload key=SNPNT5JFJOUS upload=2HYAY20O4MAK tag=PIKRMK0OP584 type=write size=3421347 2024-05-09T20:16:53.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.396+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~6PAM5998TGMT.0 size=92725 2024-05-09T20:16:53.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.396+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=6PAM5998TGMT tag=B5C2MC8PJBFL type=write size=1141301 2024-05-09T20:16:53.391 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.400+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=69L4O20XXRYQ type=del 2024-05-09T20:16:53.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.404+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=MLA1UVQKKIH5 type=write size=470329 2024-05-09T20:16:53.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.404+0000 7fdc882e58c0 1 < completed write operation key=NNGWUSCETMH4 tag=LCU97RUC01GR type=write size=764723 2024-05-09T20:16:53.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.408+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=57V0XZ6QSV8B type=del size=4137998 2024-05-09T20:16:53.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.412+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~XCTOC7B754B1.0 size=192732 2024-05-09T20:16:53.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.416+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=XCTOC7B754B1 tag=PBFNECEMOUMT type=write size=1241308 2024-05-09T20:16:53.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.420+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~Y67PD33H9QR6.0 size=1048576 2024-05-09T20:16:53.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~Y67PD33H9QR6.1 size=128434 2024-05-09T20:16:53.416 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.424+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=Y67PD33H9QR6 tag=MC9RWX9GAS62 type=write size=2225586 2024-05-09T20:16:53.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.428+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~VUKR41TE4ZQ2.0 size=628677 2024-05-09T20:16:53.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.432+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=VUKR41TE4ZQ2 tag=U4O3MD3OJ78O type=write size=1677253 2024-05-09T20:16:53.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.432+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=R1Q0S7SSSR0D type=del size=2840316 2024-05-09T20:16:53.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.436+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~WHHG9NF66KM3.0 size=1048576 2024-05-09T20:16:53.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.436+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~WHHG9NF66KM3.1 size=252862 2024-05-09T20:16:53.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.444+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=WHHG9NF66KM3 tag=C3WI8JCUDKD2 type=write size=2350014 2024-05-09T20:16:53.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.444+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=5XQEFL3BDMHF type=write size=47896 2024-05-09T20:16:53.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.448+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~TQUOS8R2Y7PV.0 size=1048576 2024-05-09T20:16:53.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.452+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~TQUOS8R2Y7PV.1 size=507410 2024-05-09T20:16:53.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.456+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=TQUOS8R2Y7PV tag=DT5D73L8HFRD type=write size=2604562 2024-05-09T20:16:53.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.460+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=TGQJV1W08O8Y type=del size=788846 2024-05-09T20:16:53.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.460+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~8793WPQ030PV.0 size=1048576 2024-05-09T20:16:53.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.464+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~8793WPQ030PV.1 size=1048576 2024-05-09T20:16:53.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.468+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~8793WPQ030PV.2 size=150010 2024-05-09T20:16:53.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.472+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=8793WPQ030PV tag=45S3VENOQ24K type=write size=3295738 2024-05-09T20:16:53.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.476+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~ZRTVGZAJM3YV.0 size=779075 2024-05-09T20:16:53.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.476+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=ZRTVGZAJM3YV tag=1YW81KY1R60Z type=write size=1827651 2024-05-09T20:16:53.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.480+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~CYUTTCQVTFB7.0 size=1048576 2024-05-09T20:16:53.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.480+0000 7fdc882e58c0 1 < canceled part key=_multipart_T5EL7X0P450L.2~CYUTTCQVTFB7.1 size=153466 2024-05-09T20:16:53.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.484+0000 7fdc882e58c0 1 < canceled multipart upload key=T5EL7X0P450L upload=CYUTTCQVTFB7 tag=YJAM0X5E5KJL type=write size=2250618 2024-05-09T20:16:53.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.488+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=2M3OXJZH5FVC type=write size=44838 2024-05-09T20:16:53.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.492+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=RHA1EFE0177U type=del 2024-05-09T20:16:53.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.496+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=BNHYG9G9730V type=write size=550895 2024-05-09T20:16:53.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.500+0000 7fdc882e58c0 1 < suggested update operation key=BS0T1OQOBS67 tag=QNYACYQ4PWPI type=write size=22668 2024-05-09T20:16:53.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.504+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=D245A7PUZ1F8 type=del size=1566348 2024-05-09T20:16:53.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.512+0000 7fdc882e58c0 1 < suggested remove operation key=NNGWUSCETMH4 tag=WOSZ1FTFS58L type=del 2024-05-09T20:16:53.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.516+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=YYU4JKUR11OW type=del size=1873853 2024-05-09T20:16:53.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.516+0000 7fdc882e58c0 1 < canceled part key=_multipart_JS90W6PIREMD.2~L3VZLBY2JCZD.0 size=1048576 2024-05-09T20:16:53.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.516+0000 7fdc882e58c0 1 < canceled part key=_multipart_JS90W6PIREMD.2~L3VZLBY2JCZD.1 size=751842 2024-05-09T20:16:53.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.520+0000 7fdc882e58c0 1 < canceled multipart upload key=JS90W6PIREMD upload=L3VZLBY2JCZD tag=KPRZ2LND7G1P type=write size=2848994 2024-05-09T20:16:53.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.524+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=7AXJ9Z71KX9V type=del size=1088010 2024-05-09T20:16:53.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.528+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~K7IHOHP3Y6CX.0 size=85105 2024-05-09T20:16:53.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.532+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=K7IHOHP3Y6CX tag=VCUY3GEMD5FG type=write size=1133681 2024-05-09T20:16:53.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.536+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=2OAP1PTTG3OG type=del size=3250786 2024-05-09T20:16:53.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.540+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~WLKFGHUWN91H.0 size=643148 2024-05-09T20:16:53.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.544+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=WLKFGHUWN91H tag=KX0S8D0KOCZ1 type=write size=1691724 2024-05-09T20:16:53.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.548+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~VUCUIKTI9RMH.0 size=1048576 2024-05-09T20:16:53.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.552+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~VUCUIKTI9RMH.1 size=601608 2024-05-09T20:16:53.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.552+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=VUCUIKTI9RMH tag=9Y4WH26G9KTT type=write size=2698760 2024-05-09T20:16:53.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.552+0000 7fdc882e58c0 1 < canceled operation key=P2DNA4DGI54P tag=UDSY9L1EN5QX type=del size=337950 2024-05-09T20:16:53.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.556+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=GL6L0ICZNCXZ type=del size=1311754 2024-05-09T20:16:53.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.556+0000 7fdc882e58c0 1 < canceled part key=_multipart_J2Q86HKRMAKE.2~6PAM5998TGMT.0 size=92725 2024-05-09T20:16:53.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.560+0000 7fdc882e58c0 1 < canceled multipart upload key=J2Q86HKRMAKE upload=6PAM5998TGMT tag=B5C2MC8PJBFL type=write size=1141301 2024-05-09T20:16:53.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.564+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=M1FZRFG4AM5O type=del size=3785800 2024-05-09T20:16:53.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.568+0000 7fdc882e58c0 1 < completed write operation key=FNB9B18LIA8L tag=MLA1UVQKKIH5 type=write size=470329 2024-05-09T20:16:53.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.568+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=VHE4N736HZMI type=del size=1669589 2024-05-09T20:16:53.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.576+0000 7fdc882e58c0 1 < suggested update operation key=5NEOR8OOFAFZ tag=57V0XZ6QSV8B type=del size=4137998 2024-05-09T20:16:53.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.580+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=LR9BC81Q7OMH type=del size=167839 2024-05-09T20:16:53.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.580+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~Y67PD33H9QR6.0 size=1048576 2024-05-09T20:16:53.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.584+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~Y67PD33H9QR6.1 size=128434 2024-05-09T20:16:53.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.588+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=Y67PD33H9QR6 tag=MC9RWX9GAS62 type=write size=2225586 2024-05-09T20:16:53.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.592+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~XNTADSGJI37A.0 size=1048576 2024-05-09T20:16:53.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.596+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~XNTADSGJI37A.1 size=1048576 2024-05-09T20:16:53.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.600+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~XNTADSGJI37A.2 size=226668 2024-05-09T20:16:53.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.600+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=XNTADSGJI37A tag=WDQ6O7KFU4ZT type=write size=3372396 2024-05-09T20:16:53.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.604+0000 7fdc882e58c0 1 < suggested update operation key=T6T3IOW9VPA0 tag=R1Q0S7SSSR0D type=del size=2840316 2024-05-09T20:16:53.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~YRVPJW9UFHAP.0 size=27613 2024-05-09T20:16:53.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.612+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=YRVPJW9UFHAP tag=DCJ1OP58Z7R2 type=write size=1076189 2024-05-09T20:16:53.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.612+0000 7fdc882e58c0 -1 < failed to complete operation key=5NEOR8OOFAFZ tag=5XQEFL3BDMHF type=write size=47896: (22) Invalid argument 2024-05-09T20:16:53.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.616+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=F29HH7CZU0EU type=write size=399635 2024-05-09T20:16:53.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.620+0000 7fdc882e58c0 1 < suggested remove operation key=346HV1XQKCOZ tag=TGQJV1W08O8Y type=del 2024-05-09T20:16:53.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.624+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~850UYJ3RMCYD.0 size=688293 2024-05-09T20:16:53.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.624+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=850UYJ3RMCYD tag=ICKAXQCKMXF2 type=write size=1736869 2024-05-09T20:16:53.621 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.628+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~ZRTVGZAJM3YV.0 size=779075 2024-05-09T20:16:53.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.632+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=ZRTVGZAJM3YV tag=1YW81KY1R60Z type=write size=1827651 2024-05-09T20:16:53.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.636+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=EGIXB4PE6KK8 type=del size=1277941 2024-05-09T20:16:53.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.640+0000 7fdc882e58c0 -1 < failed to complete operation key=NNGWUSCETMH4 tag=2M3OXJZH5FVC type=write size=44838: (22) Invalid argument 2024-05-09T20:16:53.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~AKIURGQ4XAT0.0 size=1048576 2024-05-09T20:16:53.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.648+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~AKIURGQ4XAT0.1 size=1048576 2024-05-09T20:16:53.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.648+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~AKIURGQ4XAT0.2 size=1002538 2024-05-09T20:16:53.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.648+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=AKIURGQ4XAT0 tag=KZC3FLWDH73H type=write size=4148266 2024-05-09T20:16:53.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.652+0000 7fdc882e58c0 1 < canceled operation key=NBAECV9L85UK tag=BNHYG9G9730V type=write size=550895 2024-05-09T20:16:53.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.656+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=75N1W7W7608E type=del size=3025672 2024-05-09T20:16:53.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.656+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=D245A7PUZ1F8 type=del 2024-05-09T20:16:53.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.660+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=219T9DU69CJG type=del size=2786346 2024-05-09T20:16:53.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.664+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=YYU4JKUR11OW type=del 2024-05-09T20:16:53.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.668+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=TT9EU880B7TL type=del size=3737212 2024-05-09T20:16:53.664 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.672+0000 7fdc882e58c0 1 < suggested update operation key=T6T3IOW9VPA0 tag=7AXJ9Z71KX9V type=del size=1088010 2024-05-09T20:16:53.667 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.676+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=HOGTDCY8F4N4 type=del size=546753 2024-05-09T20:16:53.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.680+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=2OAP1PTTG3OG type=del 2024-05-09T20:16:53.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.684+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=BZJ26BYV23GX type=write size=657893 2024-05-09T20:16:53.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.684+0000 7fdc882e58c0 1 < canceled part key=_multipart_G0DX1F257RLD.2~VUCUIKTI9RMH.0 size=1048576 2024-05-09T20:16:53.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.688+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~VUCUIKTI9RMH.1 size=601608 2024-05-09T20:16:53.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.692+0000 7fdc882e58c0 1 < canceled multipart upload key=G0DX1F257RLD upload=VUCUIKTI9RMH tag=9Y4WH26G9KTT type=write size=2698760 2024-05-09T20:16:53.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.696+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=TWXWGZJ85VQ9 type=del size=1739823 2024-05-09T20:16:53.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.700+0000 7fdc882e58c0 1 < canceled operation key=BS0T1OQOBS67 tag=GL6L0ICZNCXZ type=del size=1311754 2024-05-09T20:16:53.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.704+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~4R21BXWELRID.0 size=1048576 2024-05-09T20:16:53.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.708+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~4R21BXWELRID.1 size=978742 2024-05-09T20:16:53.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.708+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=4R21BXWELRID tag=E4194EU7N2NS type=write size=3075894 2024-05-09T20:16:53.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.712+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=M1FZRFG4AM5O type=del 2024-05-09T20:16:53.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.720+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~MEL1UFEVPATM.0 size=1048576 2024-05-09T20:16:53.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.720+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~MEL1UFEVPATM.1 size=489430 2024-05-09T20:16:53.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.720+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=MEL1UFEVPATM tag=NFTW7VGJBKA1 type=write size=2586582 2024-05-09T20:16:53.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.724+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=VHE4N736HZMI type=del 2024-05-09T20:16:53.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.728+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=ADMVMLJ4U0ZS type=del size=327511 2024-05-09T20:16:53.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.732+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=LR9BC81Q7OMH type=del 2024-05-09T20:16:53.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~RH1VYM689YWV.0 size=1048576 2024-05-09T20:16:53.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.740+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~RH1VYM689YWV.1 size=191607 2024-05-09T20:16:53.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.740+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=RH1VYM689YWV tag=6TCBWWU4F586 type=write size=2288759 2024-05-09T20:16:53.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.744+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~XNTADSGJI37A.0 size=1048576 2024-05-09T20:16:53.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.744+0000 7fdc882e58c0 1 < canceled part key=_multipart_T6T3IOW9VPA0.2~XNTADSGJI37A.1 size=1048576 2024-05-09T20:16:53.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.748+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~XNTADSGJI37A.2 size=226668 2024-05-09T20:16:53.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.752+0000 7fdc882e58c0 1 < canceled multipart upload key=T6T3IOW9VPA0 upload=XNTADSGJI37A tag=WDQ6O7KFU4ZT type=write size=3372396 2024-05-09T20:16:53.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.756+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=4GKDJVNMWGW5 type=del size=2201995 2024-05-09T20:16:53.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.760+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~YRVPJW9UFHAP.0 size=27613 2024-05-09T20:16:53.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.764+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=YRVPJW9UFHAP tag=DCJ1OP58Z7R2 type=write size=1076189 2024-05-09T20:16:53.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.768+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~0NZ4X31I3LIT.0 size=1048576 2024-05-09T20:16:53.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.772+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~0NZ4X31I3LIT.1 size=1048576 2024-05-09T20:16:53.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.776+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~0NZ4X31I3LIT.2 size=822988 2024-05-09T20:16:53.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.776+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=0NZ4X31I3LIT tag=AO98QL84Z4FX type=write size=3968716 2024-05-09T20:16:53.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.776+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=F29HH7CZU0EU type=write size=399635 2024-05-09T20:16:53.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~BX68Y6FYTZLL.0 size=699290 2024-05-09T20:16:53.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.784+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=BX68Y6FYTZLL tag=LCU3MQB7YIC7 type=write size=1747866 2024-05-09T20:16:53.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.788+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~850UYJ3RMCYD.0 size=688293 2024-05-09T20:16:53.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.792+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=850UYJ3RMCYD tag=ICKAXQCKMXF2 type=write size=1736869 2024-05-09T20:16:53.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.796+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=K6PX119W2DHX type=write size=555509 2024-05-09T20:16:53.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.800+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=EGIXB4PE6KK8 type=del 2024-05-09T20:16:53.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.804+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=08QZR11G9LBA type=del size=2903292 2024-05-09T20:16:53.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.804+0000 7fdc882e58c0 1 < canceled part key=_multipart_FNB9B18LIA8L.2~AKIURGQ4XAT0.0 size=1048576 2024-05-09T20:16:53.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.808+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~AKIURGQ4XAT0.1 size=1048576 2024-05-09T20:16:53.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.808+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~AKIURGQ4XAT0.2 size=1002538 2024-05-09T20:16:53.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.812+0000 7fdc882e58c0 1 < canceled multipart upload key=FNB9B18LIA8L upload=AKIURGQ4XAT0 tag=KZC3FLWDH73H type=write size=4148266 2024-05-09T20:16:53.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.816+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=QZ4CM0JTNL8P type=del size=1990367 2024-05-09T20:16:53.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.820+0000 7fdc882e58c0 1 < canceled operation key=O07UHAIJZF0R tag=75N1W7W7608E type=del size=3025672 2024-05-09T20:16:53.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.824+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~DRIH3U3V7XKY.0 size=317859 2024-05-09T20:16:53.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.824+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=DRIH3U3V7XKY tag=A2OS96IVH6UE type=write size=1366435 2024-05-09T20:16:53.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.828+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=219T9DU69CJG type=del 2024-05-09T20:16:53.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.832+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=5DGFR0U2C228 type=del size=1866214 2024-05-09T20:16:53.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.836+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=TT9EU880B7TL type=del 2024-05-09T20:16:53.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.840+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~MS8C78F5232B.0 size=1048576 2024-05-09T20:16:53.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.844+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~MS8C78F5232B.1 size=53136 2024-05-09T20:16:53.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.844+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=MS8C78F5232B tag=KAHEM6AB9LPN type=write size=2150288 2024-05-09T20:16:53.839 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.848+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=HOGTDCY8F4N4 type=del 2024-05-09T20:16:53.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.852+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~UCYBB3PF4G9Y.0 size=1048576 2024-05-09T20:16:53.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.856+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~UCYBB3PF4G9Y.1 size=1048576 2024-05-09T20:16:53.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.856+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~UCYBB3PF4G9Y.2 size=229762 2024-05-09T20:16:53.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.856+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=UCYBB3PF4G9Y tag=BPE2MUYP6OJE type=write size=3375490 2024-05-09T20:16:53.853 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.860+0000 7fdc882e58c0 1 < completed write operation key=EK78CNKUHNKO tag=BZJ26BYV23GX type=write size=657893 2024-05-09T20:16:53.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.868+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~AW0AMZWBHKLU.0 size=58550 2024-05-09T20:16:53.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.868+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=AW0AMZWBHKLU tag=CRWWNP0HSFN7 type=write size=1107126 2024-05-09T20:16:53.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.872+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=TWXWGZJ85VQ9 type=del 2024-05-09T20:16:53.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.872+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=Q0CHYW2RQDH8 type=del size=186910 2024-05-09T20:16:53.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.876+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~4R21BXWELRID.0 size=1048576 2024-05-09T20:16:53.870 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.880+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~4R21BXWELRID.1 size=978742 2024-05-09T20:16:53.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.884+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=4R21BXWELRID tag=E4194EU7N2NS type=write size=3075894 2024-05-09T20:16:53.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.888+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~F1BZU75RX2DH.0 size=1048576 2024-05-09T20:16:53.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.892+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~F1BZU75RX2DH.1 size=675351 2024-05-09T20:16:53.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.892+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=F1BZU75RX2DH tag=7CTMAKQ52ZVR type=write size=2772503 2024-05-09T20:16:53.886 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.896+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~MEL1UFEVPATM.0 size=1048576 2024-05-09T20:16:53.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.896+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~MEL1UFEVPATM.1 size=489430 2024-05-09T20:16:53.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.900+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=MEL1UFEVPATM tag=NFTW7VGJBKA1 type=write size=2586582 2024-05-09T20:16:53.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.904+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=SBCOYIFJB6QE type=write size=72160 2024-05-09T20:16:53.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.908+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=ADMVMLJ4U0ZS type=del 2024-05-09T20:16:53.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.912+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~1JJQ76ZQXOM0.0 size=1048576 2024-05-09T20:16:53.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.916+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~1JJQ76ZQXOM0.1 size=1048576 2024-05-09T20:16:53.909 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.916+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~1JJQ76ZQXOM0.2 size=300690 2024-05-09T20:16:53.909 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.916+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=1JJQ76ZQXOM0 tag=YLYPMZ9PCEUI type=write size=3446418 2024-05-09T20:16:53.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.920+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~RH1VYM689YWV.0 size=1048576 2024-05-09T20:16:53.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.924+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~RH1VYM689YWV.1 size=191607 2024-05-09T20:16:53.922 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.928+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=RH1VYM689YWV tag=6TCBWWU4F586 type=write size=2288759 2024-05-09T20:16:53.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.932+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~2D7Y30NC8E3T.0 size=1048576 2024-05-09T20:16:53.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~2D7Y30NC8E3T.1 size=1048576 2024-05-09T20:16:53.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~2D7Y30NC8E3T.2 size=677122 2024-05-09T20:16:53.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.936+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=2D7Y30NC8E3T tag=270WFD7OL83Z type=write size=3822850 2024-05-09T20:16:53.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.940+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=4GKDJVNMWGW5 type=del 2024-05-09T20:16:53.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.944+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~Z75CPZC8TLQA.0 size=11926 2024-05-09T20:16:53.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.944+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=Z75CPZC8TLQA tag=UCK5RGDHYR75 type=write size=1060502 2024-05-09T20:16:53.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.948+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~0NZ4X31I3LIT.0 size=1048576 2024-05-09T20:16:53.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.952+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~0NZ4X31I3LIT.1 size=1048576 2024-05-09T20:16:53.945 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.952+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~0NZ4X31I3LIT.2 size=822988 2024-05-09T20:16:53.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.956+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=0NZ4X31I3LIT tag=AO98QL84Z4FX type=write size=3968716 2024-05-09T20:16:53.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.964+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~XIO3J0H0RCBW.0 size=1048576 2024-05-09T20:16:53.957 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.964+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~XIO3J0H0RCBW.1 size=1048576 2024-05-09T20:16:53.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.968+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~XIO3J0H0RCBW.2 size=503731 2024-05-09T20:16:53.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.968+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=XIO3J0H0RCBW tag=E6Y218BOYHFU type=write size=3649459 2024-05-09T20:16:53.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.972+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~BX68Y6FYTZLL.0 size=699290 2024-05-09T20:16:53.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.976+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=BX68Y6FYTZLL tag=LCU3MQB7YIC7 type=write size=1747866 2024-05-09T20:16:53.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.980+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=EL0EVMYEL8NQ type=del size=652876 2024-05-09T20:16:53.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.980+0000 7fdc882e58c0 1 < canceled operation key=8FFZZDURUWM1 tag=K6PX119W2DHX type=write size=555509 2024-05-09T20:16:53.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.988+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~5CM113V1I6AL.0 size=824921 2024-05-09T20:16:53.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.988+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=5CM113V1I6AL tag=51QW7VDRM9JT type=write size=1873497 2024-05-09T20:16:53.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.992+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=08QZR11G9LBA type=del 2024-05-09T20:16:53.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.992+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=ZBOLP0DGFMYK type=write size=828639 2024-05-09T20:16:53.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:53.996+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=QZ4CM0JTNL8P type=del 2024-05-09T20:16:53.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.000+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=742EJOJ92XRE type=del size=3189124 2024-05-09T20:16:53.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.004+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~DRIH3U3V7XKY.0 size=317859 2024-05-09T20:16:53.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.008+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=DRIH3U3V7XKY tag=A2OS96IVH6UE type=write size=1366435 2024-05-09T20:16:54.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.012+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~7A7D615QLFCP.0 size=1048576 2024-05-09T20:16:54.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~7A7D615QLFCP.1 size=597225 2024-05-09T20:16:54.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.016+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=7A7D615QLFCP tag=QYFBNUVKGXKP type=write size=2694377 2024-05-09T20:16:54.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.016+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=5DGFR0U2C228 type=del 2024-05-09T20:16:54.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.024+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=HB9H7EFPX211 type=write size=357448 2024-05-09T20:16:54.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.028+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~MS8C78F5232B.0 size=1048576 2024-05-09T20:16:54.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.028+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~MS8C78F5232B.1 size=53136 2024-05-09T20:16:54.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.036+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=MS8C78F5232B tag=KAHEM6AB9LPN type=write size=2150288 2024-05-09T20:16:54.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.036+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=QYRKY6NMQIVG type=del size=977667 2024-05-09T20:16:54.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.040+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~UCYBB3PF4G9Y.0 size=1048576 2024-05-09T20:16:54.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.044+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~UCYBB3PF4G9Y.1 size=1048576 2024-05-09T20:16:54.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.044+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~UCYBB3PF4G9Y.2 size=229762 2024-05-09T20:16:54.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.048+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=UCYBB3PF4G9Y tag=BPE2MUYP6OJE type=write size=3375490 2024-05-09T20:16:54.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.056+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~B1U303B39L14.0 size=934290 2024-05-09T20:16:54.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.056+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=B1U303B39L14 tag=N3TZGG8T4PBB type=write size=1982866 2024-05-09T20:16:54.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.056+0000 7fdc882e58c0 1 < canceled part key=_multipart_J2Q86HKRMAKE.2~AW0AMZWBHKLU.0 size=58550 2024-05-09T20:16:54.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.060+0000 7fdc882e58c0 1 < canceled multipart upload key=J2Q86HKRMAKE upload=AW0AMZWBHKLU tag=CRWWNP0HSFN7 type=write size=1107126 2024-05-09T20:16:54.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.064+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~X8CRJVD2MYX8.0 size=1048576 2024-05-09T20:16:54.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.068+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~X8CRJVD2MYX8.1 size=336525 2024-05-09T20:16:54.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.068+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=X8CRJVD2MYX8 tag=B326QR9MA4GE type=write size=2433677 2024-05-09T20:16:54.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.072+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=Q0CHYW2RQDH8 type=del 2024-05-09T20:16:54.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.072+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=CFYKDDQMA6G5 type=del size=665481 2024-05-09T20:16:54.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.076+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~F1BZU75RX2DH.0 size=1048576 2024-05-09T20:16:54.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.080+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~F1BZU75RX2DH.1 size=675351 2024-05-09T20:16:54.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.084+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=F1BZU75RX2DH tag=7CTMAKQ52ZVR type=write size=2772503 2024-05-09T20:16:54.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.088+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=B84R8V3V4QT6 type=write size=829749 2024-05-09T20:16:54.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.092+0000 7fdc882e58c0 1 < completed write operation key=OGVA2778MJWU tag=SBCOYIFJB6QE type=write size=72160 2024-05-09T20:16:54.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.096+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=SQBWNYXK1O0H type=del size=381919 2024-05-09T20:16:54.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.096+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~1JJQ76ZQXOM0.0 size=1048576 2024-05-09T20:16:54.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.100+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~1JJQ76ZQXOM0.1 size=1048576 2024-05-09T20:16:54.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.104+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~1JJQ76ZQXOM0.2 size=300690 2024-05-09T20:16:54.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.108+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=1JJQ76ZQXOM0 tag=YLYPMZ9PCEUI type=write size=3446418 2024-05-09T20:16:54.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.112+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=H4D9E536UWL4 type=del size=3070302 2024-05-09T20:16:54.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.116+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~2D7Y30NC8E3T.0 size=1048576 2024-05-09T20:16:54.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.116+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~2D7Y30NC8E3T.1 size=1048576 2024-05-09T20:16:54.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.120+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~2D7Y30NC8E3T.2 size=677122 2024-05-09T20:16:54.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.128+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=2D7Y30NC8E3T tag=270WFD7OL83Z type=write size=3822850 2024-05-09T20:16:54.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.128+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=S96B2272OCUW type=del size=1601501 2024-05-09T20:16:54.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.132+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~Z75CPZC8TLQA.0 size=11926 2024-05-09T20:16:54.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.140+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=Z75CPZC8TLQA tag=UCK5RGDHYR75 type=write size=1060502 2024-05-09T20:16:54.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.144+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=HMRSK3SIO30V type=del size=93578 2024-05-09T20:16:54.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.148+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~XIO3J0H0RCBW.0 size=1048576 2024-05-09T20:16:54.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.148+0000 7fdc882e58c0 1 < canceled part key=_multipart_EK78CNKUHNKO.2~XIO3J0H0RCBW.1 size=1048576 2024-05-09T20:16:54.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.152+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~XIO3J0H0RCBW.2 size=503731 2024-05-09T20:16:54.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.156+0000 7fdc882e58c0 1 < canceled multipart upload key=EK78CNKUHNKO upload=XIO3J0H0RCBW tag=E6Y218BOYHFU type=write size=3649459 2024-05-09T20:16:54.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.160+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=WL03LGNRNK8R type=del size=811259 2024-05-09T20:16:54.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.164+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=EL0EVMYEL8NQ type=del 2024-05-09T20:16:54.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.172+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~PWFIZE8MQ2Q7.0 size=735146 2024-05-09T20:16:54.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.172+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=PWFIZE8MQ2Q7 tag=VEP0FRZELFLN type=write size=1783722 2024-05-09T20:16:54.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.176+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~5CM113V1I6AL.0 size=824921 2024-05-09T20:16:54.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.180+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=5CM113V1I6AL tag=51QW7VDRM9JT type=write size=1873497 2024-05-09T20:16:54.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.184+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=RYM1LWOA771E type=del size=3296767 2024-05-09T20:16:54.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.188+0000 7fdc882e58c0 1 < completed write operation key=5NEOR8OOFAFZ tag=ZBOLP0DGFMYK type=write size=828639 2024-05-09T20:16:54.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.196+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~YE5HQGF0Y8S2.0 size=374388 2024-05-09T20:16:54.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.196+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=YE5HQGF0Y8S2 tag=FP8JOCUIPY8S type=write size=1422964 2024-05-09T20:16:54.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.200+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=742EJOJ92XRE type=del 2024-05-09T20:16:54.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.208+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~W4Y3TKF5D0NC.0 size=1048576 2024-05-09T20:16:54.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.208+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~W4Y3TKF5D0NC.1 size=984308 2024-05-09T20:16:54.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.208+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=W4Y3TKF5D0NC tag=73V5XG3HV7HV type=write size=3081460 2024-05-09T20:16:54.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.212+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~7A7D615QLFCP.0 size=1048576 2024-05-09T20:16:54.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.212+0000 7fdc882e58c0 1 < canceled part key=_multipart_IZPPFLWVPRKO.2~7A7D615QLFCP.1 size=597225 2024-05-09T20:16:54.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.220+0000 7fdc882e58c0 1 < canceled multipart upload key=IZPPFLWVPRKO upload=7A7D615QLFCP tag=QYFBNUVKGXKP type=write size=2694377 2024-05-09T20:16:54.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.224+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=W2J1N2W8R3OJ type=del size=2990252 2024-05-09T20:16:54.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.228+0000 7fdc882e58c0 1 < completed write operation key=JS90W6PIREMD tag=HB9H7EFPX211 type=write size=357448 2024-05-09T20:16:54.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.232+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=BDUGRCE6169Z type=del size=1444123 2024-05-09T20:16:54.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.236+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=QYRKY6NMQIVG type=del 2024-05-09T20:16:54.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.240+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=ZMDAGPHAU28D type=write size=625093 2024-05-09T20:16:54.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.244+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~B1U303B39L14.0 size=934290 2024-05-09T20:16:54.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.248+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=B1U303B39L14 tag=N3TZGG8T4PBB type=write size=1982866 2024-05-09T20:16:54.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.256+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~73R02OYJ1G54.0 size=1048576 2024-05-09T20:16:54.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.260+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~73R02OYJ1G54.1 size=1048576 2024-05-09T20:16:54.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.260+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~73R02OYJ1G54.2 size=1013111 2024-05-09T20:16:54.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.260+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=73R02OYJ1G54 tag=J2OYA3G5X82E type=write size=4158839 2024-05-09T20:16:54.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.264+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~X8CRJVD2MYX8.0 size=1048576 2024-05-09T20:16:54.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.268+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~X8CRJVD2MYX8.1 size=336525 2024-05-09T20:16:54.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.276+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=X8CRJVD2MYX8 tag=B326QR9MA4GE type=write size=2433677 2024-05-09T20:16:54.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.280+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=TYKC4EVOO8IW type=write size=875071 2024-05-09T20:16:54.275 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.284+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=CFYKDDQMA6G5 type=del 2024-05-09T20:16:54.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.288+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=KE18BY0OA4UL type=del size=1904915 2024-05-09T20:16:54.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.292+0000 7fdc882e58c0 1 < suggested remove operation key=VZ7XOAT899GU tag=B84R8V3V4QT6 type=write 2024-05-09T20:16:54.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.296+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=XX52G56LIJ6W type=del size=3302170 2024-05-09T20:16:54.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.304+0000 7fdc882e58c0 1 < suggested remove operation key=NBAECV9L85UK tag=SQBWNYXK1O0H type=del 2024-05-09T20:16:54.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.308+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=YVEBTY42U1KI type=write size=472123 2024-05-09T20:16:54.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.316+0000 7fdc882e58c0 1 < suggested update operation key=T6T3IOW9VPA0 tag=H4D9E536UWL4 type=del size=3070302 2024-05-09T20:16:54.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.320+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=IAI3UO49O41E type=del size=2447784 2024-05-09T20:16:54.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.324+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=S96B2272OCUW type=del 2024-05-09T20:16:54.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.328+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=9R75B5O1MOJP type=del size=486579 2024-05-09T20:16:54.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.332+0000 7fdc882e58c0 -1 < failed to complete operation key=T6T3IOW9VPA0 tag=HMRSK3SIO30V type=del size=93578: (22) Invalid argument 2024-05-09T20:16:54.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.332+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=A9NYAG7LXN5D type=del size=3771166 2024-05-09T20:16:54.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.336+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=WL03LGNRNK8R type=del 2024-05-09T20:16:54.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.344+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~B9MIYRW3UWRJ.0 size=1048576 2024-05-09T20:16:54.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.348+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~B9MIYRW3UWRJ.1 size=446422 2024-05-09T20:16:54.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.348+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=B9MIYRW3UWRJ tag=FU48THE9CYR7 type=write size=2543574 2024-05-09T20:16:54.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.352+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~PWFIZE8MQ2Q7.0 size=735146 2024-05-09T20:16:54.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.360+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=PWFIZE8MQ2Q7 tag=VEP0FRZELFLN type=write size=1783722 2024-05-09T20:16:54.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.364+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=GN5ZKW55R8XY type=write size=700417 2024-05-09T20:16:54.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.368+0000 7fdc882e58c0 1 < suggested update operation key=TWWOL5D8C1MQ tag=RYM1LWOA771E type=del size=3296767 2024-05-09T20:16:54.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.372+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=8WCHD9FCPBG6 type=write size=528541 2024-05-09T20:16:54.370 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.376+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~YE5HQGF0Y8S2.0 size=374388 2024-05-09T20:16:54.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.384+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=YE5HQGF0Y8S2 tag=FP8JOCUIPY8S type=write size=1422964 2024-05-09T20:16:54.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.388+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=1U663V226DPO type=del size=367112 2024-05-09T20:16:54.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.392+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~W4Y3TKF5D0NC.0 size=1048576 2024-05-09T20:16:54.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.396+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~W4Y3TKF5D0NC.1 size=984308 2024-05-09T20:16:54.392 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.400+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=W4Y3TKF5D0NC tag=73V5XG3HV7HV type=write size=3081460 2024-05-09T20:16:54.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.404+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~5BT5F9GTP70X.0 size=1048576 2024-05-09T20:16:54.400 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.408+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~5BT5F9GTP70X.1 size=1048576 2024-05-09T20:16:54.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.412+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~5BT5F9GTP70X.2 size=23305 2024-05-09T20:16:54.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.412+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=5BT5F9GTP70X tag=3HSSTR1P3LNO type=write size=3169033 2024-05-09T20:16:54.405 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.412+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=W2J1N2W8R3OJ type=del 2024-05-09T20:16:54.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.416+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=FJ1KYWOSWX0F type=del size=732866 2024-05-09T20:16:54.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.420+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=BDUGRCE6169Z type=del 2024-05-09T20:16:54.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.424+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=IX89TTB591ZL type=del size=4175629 2024-05-09T20:16:54.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.428+0000 7fdc882e58c0 1 < completed write operation key=P2DNA4DGI54P tag=ZMDAGPHAU28D type=write size=625093 2024-05-09T20:16:54.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.428+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=YZN8ZWSRIKG9 type=write size=541123 2024-05-09T20:16:54.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.432+0000 7fdc882e58c0 1 < canceled part key=_multipart_346HV1XQKCOZ.2~73R02OYJ1G54.0 size=1048576 2024-05-09T20:16:54.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.432+0000 7fdc882e58c0 1 < canceled part key=_multipart_346HV1XQKCOZ.2~73R02OYJ1G54.1 size=1048576 2024-05-09T20:16:54.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.432+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~73R02OYJ1G54.2 size=1013111 2024-05-09T20:16:54.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.436+0000 7fdc882e58c0 1 < canceled multipart upload key=346HV1XQKCOZ upload=73R02OYJ1G54 tag=J2OYA3G5X82E type=write size=4158839 2024-05-09T20:16:54.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.440+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=ML9EKGF9G5GV type=del size=1316924 2024-05-09T20:16:54.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.444+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=TYKC4EVOO8IW type=write size=875071 2024-05-09T20:16:54.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.444+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=WE7HAYJ02KQA type=del size=1234201 2024-05-09T20:16:54.440 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.448+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=KE18BY0OA4UL type=del 2024-05-09T20:16:54.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.456+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~ZAJCQ3S8GSL2.0 size=1048576 2024-05-09T20:16:54.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.456+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~ZAJCQ3S8GSL2.1 size=1048576 2024-05-09T20:16:54.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.460+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~ZAJCQ3S8GSL2.2 size=46857 2024-05-09T20:16:54.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.460+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=ZAJCQ3S8GSL2 tag=79WC47GG8IR8 type=write size=3192585 2024-05-09T20:16:54.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.464+0000 7fdc882e58c0 1 < canceled operation key=NNGWUSCETMH4 tag=XX52G56LIJ6W type=del size=3302170 2024-05-09T20:16:54.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.464+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=15HNCIJW6CN5 type=del size=3132749 2024-05-09T20:16:54.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.468+0000 7fdc882e58c0 1 < completed write operation key=CNV7NHP27JZ7 tag=YVEBTY42U1KI type=write size=472123 2024-05-09T20:16:54.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.476+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~1R0660E112MS.0 size=1016648 2024-05-09T20:16:54.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.476+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=1R0660E112MS tag=O2XGB840MRH5 type=write size=2065224 2024-05-09T20:16:54.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.480+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=IAI3UO49O41E type=del 2024-05-09T20:16:54.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.484+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=4UOVGCV4SHMQ type=del size=842575 2024-05-09T20:16:54.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.488+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=9R75B5O1MOJP type=del 2024-05-09T20:16:54.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.492+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~SMKFTVF9G9WF.0 size=1048576 2024-05-09T20:16:54.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.496+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~SMKFTVF9G9WF.1 size=1048576 2024-05-09T20:16:54.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.500+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~SMKFTVF9G9WF.2 size=626904 2024-05-09T20:16:54.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.500+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=SMKFTVF9G9WF tag=8X175SFL4QWD type=write size=3772632 2024-05-09T20:16:54.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.500+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=A9NYAG7LXN5D type=del 2024-05-09T20:16:54.497 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.504+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=93CBF4R0KCBJ type=write size=810454 2024-05-09T20:16:54.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.508+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~B9MIYRW3UWRJ.0 size=1048576 2024-05-09T20:16:54.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.512+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~B9MIYRW3UWRJ.1 size=446422 2024-05-09T20:16:54.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.516+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=B9MIYRW3UWRJ tag=FU48THE9CYR7 type=write size=2543574 2024-05-09T20:16:54.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.524+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~7O1A2MQ15VGK.0 size=587027 2024-05-09T20:16:54.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.524+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=7O1A2MQ15VGK tag=I6XDBNCP4ZDF type=write size=1635603 2024-05-09T20:16:54.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.528+0000 7fdc882e58c0 1 < suggested remove operation key=SPXHCWZICCWY tag=GN5ZKW55R8XY type=write 2024-05-09T20:16:54.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.536+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~S9QX661QV63M.0 size=1048576 2024-05-09T20:16:54.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.540+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~S9QX661QV63M.1 size=827615 2024-05-09T20:16:54.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.540+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=S9QX661QV63M tag=3T3WD2ZLKHCW type=write size=2924767 2024-05-09T20:16:54.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.540+0000 7fdc882e58c0 1 < completed write operation key=QM53IENH9CX4 tag=8WCHD9FCPBG6 type=write size=528541 2024-05-09T20:16:54.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.544+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=NITXIL5M2FKA type=del size=37923 2024-05-09T20:16:54.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.548+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=1U663V226DPO type=del 2024-05-09T20:16:54.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.552+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=24LGDSF2R02Z type=write size=42797 2024-05-09T20:16:54.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.556+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~5BT5F9GTP70X.0 size=1048576 2024-05-09T20:16:54.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.556+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~5BT5F9GTP70X.1 size=1048576 2024-05-09T20:16:54.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.560+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~5BT5F9GTP70X.2 size=23305 2024-05-09T20:16:54.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.564+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=5BT5F9GTP70X tag=3HSSTR1P3LNO type=write size=3169033 2024-05-09T20:16:54.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.572+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~QB52JU2V6N9A.0 size=1048576 2024-05-09T20:16:54.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.576+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~QB52JU2V6N9A.1 size=1048576 2024-05-09T20:16:54.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.576+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~QB52JU2V6N9A.2 size=226029 2024-05-09T20:16:54.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.576+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=QB52JU2V6N9A tag=YC7CPIRHO3XJ type=write size=3371757 2024-05-09T20:16:54.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.580+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=FJ1KYWOSWX0F type=del 2024-05-09T20:16:54.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.584+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=FZNEKS9EPD2N type=del size=4024312 2024-05-09T20:16:54.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.588+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=IX89TTB591ZL type=del 2024-05-09T20:16:54.583 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.592+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=PDIKZ14OCWYS type=del size=2496393 2024-05-09T20:16:54.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.596+0000 7fdc882e58c0 1 < completed write operation key=NBAECV9L85UK tag=YZN8ZWSRIKG9 type=write size=541123 2024-05-09T20:16:54.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.600+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=TYQXSZX0U9NA type=del size=436393 2024-05-09T20:16:54.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.604+0000 7fdc882e58c0 1 < suggested remove operation key=EIMPFFK4I6NZ tag=ML9EKGF9G5GV type=del 2024-05-09T20:16:54.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~X22MGFOB9MY8.0 size=379216 2024-05-09T20:16:54.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.612+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=X22MGFOB9MY8 tag=K6PAWDOGCU3Z type=write size=1427792 2024-05-09T20:16:54.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.620+0000 7fdc882e58c0 1 < suggested update operation key=5NEOR8OOFAFZ tag=WE7HAYJ02KQA type=del size=1234201 2024-05-09T20:16:54.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.628+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~PUA6COE31EV6.0 size=1048576 2024-05-09T20:16:54.621 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.628+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~PUA6COE31EV6.1 size=338992 2024-05-09T20:16:54.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.628+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=PUA6COE31EV6 tag=XCQ58CGD4FFC type=write size=2436144 2024-05-09T20:16:54.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.632+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~ZAJCQ3S8GSL2.0 size=1048576 2024-05-09T20:16:54.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.636+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~ZAJCQ3S8GSL2.1 size=1048576 2024-05-09T20:16:54.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.640+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~ZAJCQ3S8GSL2.2 size=46857 2024-05-09T20:16:54.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.644+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=ZAJCQ3S8GSL2 tag=79WC47GG8IR8 type=write size=3192585 2024-05-09T20:16:54.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.648+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=J2Z4XNHQSB1A type=del size=1781706 2024-05-09T20:16:54.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.652+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=15HNCIJW6CN5 type=del 2024-05-09T20:16:54.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.656+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=O2QMUERVK1HW type=del size=1317597 2024-05-09T20:16:54.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.660+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~1R0660E112MS.0 size=1016648 2024-05-09T20:16:54.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.668+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=1R0660E112MS tag=O2XGB840MRH5 type=write size=2065224 2024-05-09T20:16:54.667 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.676+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~JASF6Z2R4Y84.0 size=1048576 2024-05-09T20:16:54.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.680+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~JASF6Z2R4Y84.1 size=742728 2024-05-09T20:16:54.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.680+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=JASF6Z2R4Y84 tag=PTT7H1HYBMWQ type=write size=2839880 2024-05-09T20:16:54.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.680+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=4UOVGCV4SHMQ type=del 2024-05-09T20:16:54.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.688+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~1JD9XESFVZVJ.0 size=1048576 2024-05-09T20:16:54.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.692+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~1JD9XESFVZVJ.1 size=886503 2024-05-09T20:16:54.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.692+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=1JD9XESFVZVJ tag=T4AP6T9EVV7F type=write size=2983655 2024-05-09T20:16:54.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.696+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~SMKFTVF9G9WF.0 size=1048576 2024-05-09T20:16:54.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.696+0000 7fdc882e58c0 1 < canceled part key=_multipart_TQ5ZHATLNVPP.2~SMKFTVF9G9WF.1 size=1048576 2024-05-09T20:16:54.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.700+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~SMKFTVF9G9WF.2 size=626904 2024-05-09T20:16:54.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.704+0000 7fdc882e58c0 1 < canceled multipart upload key=TQ5ZHATLNVPP upload=SMKFTVF9G9WF tag=8X175SFL4QWD type=write size=3772632 2024-05-09T20:16:54.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.708+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=5TNER7YTE5CO type=del size=2737010 2024-05-09T20:16:54.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.712+0000 7fdc882e58c0 -1 < failed to complete operation key=5NEOR8OOFAFZ tag=93CBF4R0KCBJ type=write size=810454: (22) Invalid argument 2024-05-09T20:16:54.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.716+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=9RJ09Z5UVEYN type=del size=1932788 2024-05-09T20:16:54.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.720+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~7O1A2MQ15VGK.0 size=587027 2024-05-09T20:16:54.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.728+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=7O1A2MQ15VGK tag=I6XDBNCP4ZDF type=write size=1635603 2024-05-09T20:16:54.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.732+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=84CXZJROD0A8 type=write size=368426 2024-05-09T20:16:54.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.732+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~S9QX661QV63M.0 size=1048576 2024-05-09T20:16:54.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.736+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~S9QX661QV63M.1 size=827615 2024-05-09T20:16:54.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.744+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=S9QX661QV63M tag=3T3WD2ZLKHCW type=write size=2924767 2024-05-09T20:16:54.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.744+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=MC80BAQXLY17 type=del size=1877304 2024-05-09T20:16:54.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.748+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=NITXIL5M2FKA type=del 2024-05-09T20:16:54.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.756+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~E736GHVSIQL6.0 size=898331 2024-05-09T20:16:54.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.756+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=E736GHVSIQL6 tag=FGZZP7P9N23Z type=write size=1946907 2024-05-09T20:16:54.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.764+0000 7fdc882e58c0 1 < suggested remove operation key=8FFZZDURUWM1 tag=24LGDSF2R02Z type=write 2024-05-09T20:16:54.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.764+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=1BDBKCUFB33F type=write size=284750 2024-05-09T20:16:54.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.768+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~QB52JU2V6N9A.0 size=1048576 2024-05-09T20:16:54.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.772+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~QB52JU2V6N9A.1 size=1048576 2024-05-09T20:16:54.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.772+0000 7fdc882e58c0 1 < canceled part key=_multipart_AJANS5ADJO5B.2~QB52JU2V6N9A.2 size=226029 2024-05-09T20:16:54.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.776+0000 7fdc882e58c0 1 < canceled multipart upload key=AJANS5ADJO5B upload=QB52JU2V6N9A tag=YC7CPIRHO3XJ type=write size=3371757 2024-05-09T20:16:54.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~V4HC95A8VEVZ.0 size=1048576 2024-05-09T20:16:54.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~V4HC95A8VEVZ.1 size=823341 2024-05-09T20:16:54.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.784+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=V4HC95A8VEVZ tag=MJHLVMAZC7UV type=write size=2920493 2024-05-09T20:16:54.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.788+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=FZNEKS9EPD2N type=del 2024-05-09T20:16:54.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.792+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=Z7KPTKCADSCH type=write size=194468 2024-05-09T20:16:54.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.796+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=PDIKZ14OCWYS type=del 2024-05-09T20:16:54.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.800+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~0WVIAJU944D0.0 size=666189 2024-05-09T20:16:54.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.800+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=0WVIAJU944D0 tag=1JHD356A6DUG type=write size=1714765 2024-05-09T20:16:54.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.804+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=TYQXSZX0U9NA type=del 2024-05-09T20:16:54.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.812+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~AS1WKQEHDIHN.0 size=1048576 2024-05-09T20:16:54.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.812+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~AS1WKQEHDIHN.1 size=1048576 2024-05-09T20:16:54.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.816+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~AS1WKQEHDIHN.2 size=818497 2024-05-09T20:16:54.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.816+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=AS1WKQEHDIHN tag=DQ8UTESSURV7 type=write size=3964225 2024-05-09T20:16:54.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.816+0000 7fdc882e58c0 1 < canceled part key=_multipart_SPXHCWZICCWY.2~X22MGFOB9MY8.0 size=379216 2024-05-09T20:16:54.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.824+0000 7fdc882e58c0 1 < canceled multipart upload key=SPXHCWZICCWY upload=X22MGFOB9MY8 tag=K6PAWDOGCU3Z type=write size=1427792 2024-05-09T20:16:54.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.824+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=70K7LI7NXJ8Z type=del size=3799188 2024-05-09T20:16:54.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.828+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~PUA6COE31EV6.0 size=1048576 2024-05-09T20:16:54.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.832+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~PUA6COE31EV6.1 size=338992 2024-05-09T20:16:54.830 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.840+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=PUA6COE31EV6 tag=XCQ58CGD4FFC type=write size=2436144 2024-05-09T20:16:54.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.844+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=GSV7XU627BYX type=del size=3593449 2024-05-09T20:16:54.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.852+0000 7fdc882e58c0 1 < suggested update operation key=O07UHAIJZF0R tag=J2Z4XNHQSB1A type=del size=1781706 2024-05-09T20:16:54.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.856+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~PBTJCL40ZG2V.0 size=1048576 2024-05-09T20:16:54.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.860+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~PBTJCL40ZG2V.1 size=1048576 2024-05-09T20:16:54.853 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.860+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~PBTJCL40ZG2V.2 size=519546 2024-05-09T20:16:54.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.860+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=PBTJCL40ZG2V tag=G68PL2638FYE type=write size=3665274 2024-05-09T20:16:54.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.864+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=O2QMUERVK1HW type=del 2024-05-09T20:16:54.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.868+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=OZW0TK1F5V6M type=del size=3322496 2024-05-09T20:16:54.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.872+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~JASF6Z2R4Y84.0 size=1048576 2024-05-09T20:16:54.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.876+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~JASF6Z2R4Y84.1 size=742728 2024-05-09T20:16:54.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.880+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=JASF6Z2R4Y84 tag=PTT7H1HYBMWQ type=write size=2839880 2024-05-09T20:16:54.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.884+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=RIFRIW56JNUM type=del size=2641731 2024-05-09T20:16:54.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.884+0000 7fdc882e58c0 1 < canceled part key=_multipart_NYHR6GTPQYZS.2~1JD9XESFVZVJ.0 size=1048576 2024-05-09T20:16:54.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.888+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~1JD9XESFVZVJ.1 size=886503 2024-05-09T20:16:54.886 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.896+0000 7fdc882e58c0 1 < canceled multipart upload key=NYHR6GTPQYZS upload=1JD9XESFVZVJ tag=T4AP6T9EVV7F type=write size=2983655 2024-05-09T20:16:54.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.896+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=L2ZAIJ1U44EJ type=del size=2926995 2024-05-09T20:16:54.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.900+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=5TNER7YTE5CO type=del 2024-05-09T20:16:54.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.904+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=CKQ5ZRN1S6VX type=del size=935909 2024-05-09T20:16:54.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.908+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=9RJ09Z5UVEYN type=del 2024-05-09T20:16:54.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.912+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=PTXQAQ00HGJY type=del size=1021680 2024-05-09T20:16:54.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.916+0000 7fdc882e58c0 1 < completed write operation key=TQ5ZHATLNVPP tag=84CXZJROD0A8 type=write size=368426 2024-05-09T20:16:54.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.920+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~QQW9D4F2Y476.0 size=1048576 2024-05-09T20:16:54.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.924+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~QQW9D4F2Y476.1 size=695979 2024-05-09T20:16:54.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.924+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=QQW9D4F2Y476 tag=RW91E5W9E9AI type=write size=2793131 2024-05-09T20:16:54.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.928+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=MC80BAQXLY17 type=del 2024-05-09T20:16:54.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~0TQJVJWG0CJR.0 size=1048576 2024-05-09T20:16:54.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~0TQJVJWG0CJR.1 size=374774 2024-05-09T20:16:54.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.936+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=0TQJVJWG0CJR tag=SV8CVG8HY1WA type=write size=2471926 2024-05-09T20:16:54.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.940+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~E736GHVSIQL6.0 size=898331 2024-05-09T20:16:54.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.948+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=E736GHVSIQL6 tag=FGZZP7P9N23Z type=write size=1946907 2024-05-09T20:16:54.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.952+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=2T7YM6LT1EZ8 type=write size=519657 2024-05-09T20:16:54.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.956+0000 7fdc882e58c0 1 < suggested remove operation key=JS90W6PIREMD tag=1BDBKCUFB33F type=write 2024-05-09T20:16:54.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.960+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=B9SCL0EASKW2 type=del size=796121 2024-05-09T20:16:54.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.964+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~V4HC95A8VEVZ.0 size=1048576 2024-05-09T20:16:54.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.968+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~V4HC95A8VEVZ.1 size=823341 2024-05-09T20:16:54.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.972+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=V4HC95A8VEVZ tag=MJHLVMAZC7UV type=write size=2920493 2024-05-09T20:16:54.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~BCFEKUOWW0IR.0 size=1048576 2024-05-09T20:16:54.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.984+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~BCFEKUOWW0IR.1 size=1048576 2024-05-09T20:16:54.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.984+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~BCFEKUOWW0IR.2 size=38408 2024-05-09T20:16:54.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.984+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=BCFEKUOWW0IR tag=V5CAIG47N46Z type=write size=3184136 2024-05-09T20:16:54.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.988+0000 7fdc882e58c0 -1 < no bi entry to suggest for operation key=JS90W6PIREMD tag=Z7KPTKCADSCH type=write size=194468: (2) No such file or directory 2024-05-09T20:16:54.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.992+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=B2LYAHB3QNKX type=del size=2011349 2024-05-09T20:16:54.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:54.996+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~0WVIAJU944D0.0 size=666189 2024-05-09T20:16:54.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.004+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=0WVIAJU944D0 tag=1JHD356A6DUG type=write size=1714765 2024-05-09T20:16:55.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.012+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~FARLQ705S8MW.0 size=540111 2024-05-09T20:16:55.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.012+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=FARLQ705S8MW tag=JQEVOVCSNXXU type=write size=1588687 2024-05-09T20:16:55.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.016+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~AS1WKQEHDIHN.0 size=1048576 2024-05-09T20:16:55.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.016+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~AS1WKQEHDIHN.1 size=1048576 2024-05-09T20:16:55.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.020+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~AS1WKQEHDIHN.2 size=818497 2024-05-09T20:16:55.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.028+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=AS1WKQEHDIHN tag=DQ8UTESSURV7 type=write size=3964225 2024-05-09T20:16:55.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.032+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~KE1ZVOTZPQ4O.0 size=356562 2024-05-09T20:16:55.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.032+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=KE1ZVOTZPQ4O tag=QD8JHHWIJIWJ type=write size=1405138 2024-05-09T20:16:55.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.036+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=70K7LI7NXJ8Z type=del 2024-05-09T20:16:55.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.044+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~5P1U5QNK9OQR.0 size=1048576 2024-05-09T20:16:55.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.044+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~5P1U5QNK9OQR.1 size=1048576 2024-05-09T20:16:55.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.048+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~5P1U5QNK9OQR.2 size=920903 2024-05-09T20:16:55.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.048+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=5P1U5QNK9OQR tag=935MU92CRS6E type=write size=4066631 2024-05-09T20:16:55.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.052+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=GSV7XU627BYX type=del 2024-05-09T20:16:55.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.056+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=JYPPSP54O2CI type=del size=2583017 2024-05-09T20:16:55.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.060+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~PBTJCL40ZG2V.0 size=1048576 2024-05-09T20:16:55.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.060+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~PBTJCL40ZG2V.1 size=1048576 2024-05-09T20:16:55.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.064+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~PBTJCL40ZG2V.2 size=519546 2024-05-09T20:16:55.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.068+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=PBTJCL40ZG2V tag=G68PL2638FYE type=write size=3665274 2024-05-09T20:16:55.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.072+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=TO0LDV8VPDLC type=del size=117382 2024-05-09T20:16:55.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.076+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=OZW0TK1F5V6M type=del 2024-05-09T20:16:55.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.080+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=NE40FVRYW2B7 type=del size=72369 2024-05-09T20:16:55.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.084+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=RIFRIW56JNUM type=del 2024-05-09T20:16:55.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.088+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=UFU0AGEP236F type=del size=4167386 2024-05-09T20:16:55.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.092+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=L2ZAIJ1U44EJ type=del 2024-05-09T20:16:55.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.096+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=K2BFED8RB340 type=write size=181019 2024-05-09T20:16:55.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.096+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=CKQ5ZRN1S6VX type=del 2024-05-09T20:16:55.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.100+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=M22BVVGN3XR1 type=del size=1110354 2024-05-09T20:16:55.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.104+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=PTXQAQ00HGJY type=del 2024-05-09T20:16:55.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.112+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~1F45G8M3UTR1.0 size=1048576 2024-05-09T20:16:55.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.112+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~1F45G8M3UTR1.1 size=1048576 2024-05-09T20:16:55.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.116+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~1F45G8M3UTR1.2 size=810318 2024-05-09T20:16:55.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.116+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=1F45G8M3UTR1 tag=1AWBAGGBT8O3 type=write size=3956046 2024-05-09T20:16:55.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.120+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~QQW9D4F2Y476.0 size=1048576 2024-05-09T20:16:55.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.120+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~QQW9D4F2Y476.1 size=695979 2024-05-09T20:16:55.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.128+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=QQW9D4F2Y476 tag=RW91E5W9E9AI type=write size=2793131 2024-05-09T20:16:55.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.136+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~WJJBXA8YI4CD.0 size=1048576 2024-05-09T20:16:55.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.140+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~WJJBXA8YI4CD.1 size=336655 2024-05-09T20:16:55.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.140+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=WJJBXA8YI4CD tag=W7E1XE3Z4HAA type=write size=2433807 2024-05-09T20:16:55.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.144+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~0TQJVJWG0CJR.0 size=1048576 2024-05-09T20:16:55.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.148+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~0TQJVJWG0CJR.1 size=374774 2024-05-09T20:16:55.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.152+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=0TQJVJWG0CJR tag=SV8CVG8HY1WA type=write size=2471926 2024-05-09T20:16:55.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.160+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~BNK2OHW438WV.0 size=1048576 2024-05-09T20:16:55.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.164+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~BNK2OHW438WV.1 size=955728 2024-05-09T20:16:55.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.164+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=BNK2OHW438WV tag=6Y20OOWK031L type=write size=3052880 2024-05-09T20:16:55.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.168+0000 7fdc882e58c0 1 < completed write operation key=TWWOL5D8C1MQ tag=2T7YM6LT1EZ8 type=write size=519657 2024-05-09T20:16:55.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.172+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=DS20WWWWSS39 type=write size=397043 2024-05-09T20:16:55.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.176+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=B9SCL0EASKW2 type=del 2024-05-09T20:16:55.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.180+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=2ACKYWPH1X87 type=write size=750457 2024-05-09T20:16:55.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.180+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~BCFEKUOWW0IR.0 size=1048576 2024-05-09T20:16:55.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.184+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~BCFEKUOWW0IR.1 size=1048576 2024-05-09T20:16:55.179 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.188+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~BCFEKUOWW0IR.2 size=38408 2024-05-09T20:16:55.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.192+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=BCFEKUOWW0IR tag=V5CAIG47N46Z type=write size=3184136 2024-05-09T20:16:55.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.196+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=K4KAW0HEFD7F type=del size=1619097 2024-05-09T20:16:55.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.204+0000 7fdc882e58c0 1 < suggested remove operation key=AJANS5ADJO5B tag=B2LYAHB3QNKX type=del 2024-05-09T20:16:55.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.208+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=FIIIWCU0IIF3 type=del size=2492737 2024-05-09T20:16:55.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.212+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~FARLQ705S8MW.0 size=540111 2024-05-09T20:16:55.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.220+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=FARLQ705S8MW tag=JQEVOVCSNXXU type=write size=1588687 2024-05-09T20:16:55.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.224+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~X4AX63N193Z4.0 size=1048576 2024-05-09T20:16:55.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.228+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~X4AX63N193Z4.1 size=1048576 2024-05-09T20:16:55.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.232+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~X4AX63N193Z4.2 size=120416 2024-05-09T20:16:55.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.232+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=X4AX63N193Z4 tag=NPASWL4A0AD8 type=write size=3266144 2024-05-09T20:16:55.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.236+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~KE1ZVOTZPQ4O.0 size=356562 2024-05-09T20:16:55.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.240+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=KE1ZVOTZPQ4O tag=QD8JHHWIJIWJ type=write size=1405138 2024-05-09T20:16:55.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.244+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=U700W4RIISEI type=del size=2423417 2024-05-09T20:16:55.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.248+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~5P1U5QNK9OQR.0 size=1048576 2024-05-09T20:16:55.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.252+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~5P1U5QNK9OQR.1 size=1048576 2024-05-09T20:16:55.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.256+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~5P1U5QNK9OQR.2 size=920903 2024-05-09T20:16:55.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.260+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=5P1U5QNK9OQR tag=935MU92CRS6E type=write size=4066631 2024-05-09T20:16:55.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.264+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=1O8U0KAYEQZA type=del size=3599169 2024-05-09T20:16:55.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.268+0000 7fdc882e58c0 1 < canceled operation key=NBAECV9L85UK tag=JYPPSP54O2CI type=del size=2583017 2024-05-09T20:16:55.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.272+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=B6W3BZ2BCXZ7 type=write size=305893 2024-05-09T20:16:55.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.276+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=TO0LDV8VPDLC type=del 2024-05-09T20:16:55.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.280+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=3I3JC703787L type=write size=1027410 2024-05-09T20:16:55.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.280+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=NE40FVRYW2B7 type=del 2024-05-09T20:16:55.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.288+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~IH06PIAIHGUK.0 size=1048576 2024-05-09T20:16:55.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.292+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~IH06PIAIHGUK.1 size=986516 2024-05-09T20:16:55.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.292+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=IH06PIAIHGUK tag=OYHWFZFGGUM2 type=write size=3083668 2024-05-09T20:16:55.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.292+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=UFU0AGEP236F type=del 2024-05-09T20:16:55.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.300+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~TT8SBE6G2JWX.0 size=1048576 2024-05-09T20:16:55.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.304+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~TT8SBE6G2JWX.1 size=1048576 2024-05-09T20:16:55.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.308+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~TT8SBE6G2JWX.2 size=650565 2024-05-09T20:16:55.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.308+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=TT8SBE6G2JWX tag=44K4KS8L6BJ0 type=write size=3796293 2024-05-09T20:16:55.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.312+0000 7fdc882e58c0 1 < completed write operation key=TWWOL5D8C1MQ tag=K2BFED8RB340 type=write size=181019 2024-05-09T20:16:55.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.316+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~7RIWWR4HZHZ3.0 size=739998 2024-05-09T20:16:55.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.316+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=7RIWWR4HZHZ3 tag=19WSKADXC92D type=write size=1788574 2024-05-09T20:16:55.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.320+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=M22BVVGN3XR1 type=del 2024-05-09T20:16:55.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.324+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=W1Z1CC9H6V8I type=del size=1571807 2024-05-09T20:16:55.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.324+0000 7fdc882e58c0 1 < canceled part key=_multipart_O07UHAIJZF0R.2~1F45G8M3UTR1.0 size=1048576 2024-05-09T20:16:55.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.332+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~1F45G8M3UTR1.1 size=1048576 2024-05-09T20:16:55.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.332+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~1F45G8M3UTR1.2 size=810318 2024-05-09T20:16:55.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.340+0000 7fdc882e58c0 1 < canceled multipart upload key=O07UHAIJZF0R upload=1F45G8M3UTR1 tag=1AWBAGGBT8O3 type=write size=3956046 2024-05-09T20:16:55.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.344+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=LZPXLNLYSU53 type=del size=3745508 2024-05-09T20:16:55.339 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.348+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~WJJBXA8YI4CD.0 size=1048576 2024-05-09T20:16:55.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.352+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~WJJBXA8YI4CD.1 size=336655 2024-05-09T20:16:55.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.356+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=WJJBXA8YI4CD tag=W7E1XE3Z4HAA type=write size=2433807 2024-05-09T20:16:55.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.364+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~TBJENM8SASDN.0 size=1048576 2024-05-09T20:16:55.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.364+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~TBJENM8SASDN.1 size=1048576 2024-05-09T20:16:55.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.368+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~TBJENM8SASDN.2 size=500873 2024-05-09T20:16:55.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.368+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=TBJENM8SASDN tag=BHB6XXYLTWYK type=write size=3646601 2024-05-09T20:16:55.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.372+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~BNK2OHW438WV.0 size=1048576 2024-05-09T20:16:55.366 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.376+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~BNK2OHW438WV.1 size=955728 2024-05-09T20:16:55.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.380+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=BNK2OHW438WV tag=6Y20OOWK031L type=write size=3052880 2024-05-09T20:16:55.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.384+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=WW7YM4TBWYI7 type=del size=2306930 2024-05-09T20:16:55.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.388+0000 7fdc882e58c0 -1 < failed to complete operation key=AJANS5ADJO5B tag=DS20WWWWSS39 type=write size=397043: (22) Invalid argument 2024-05-09T20:16:55.384 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.392+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=U0WN3N66ZYHP type=del size=1140331 2024-05-09T20:16:55.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.396+0000 7fdc882e58c0 1 < completed write operation key=NYHR6GTPQYZS tag=2ACKYWPH1X87 type=write size=750457 2024-05-09T20:16:55.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.404+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~AMH0KMXJK7XJ.0 size=1048576 2024-05-09T20:16:55.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.404+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~AMH0KMXJK7XJ.1 size=593639 2024-05-09T20:16:55.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.404+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=AMH0KMXJK7XJ tag=F9J446AQ6SHK type=write size=2690791 2024-05-09T20:16:55.405 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.412+0000 7fdc882e58c0 1 < suggested remove operation key=346HV1XQKCOZ tag=K4KAW0HEFD7F type=del 2024-05-09T20:16:55.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.420+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~GIGLBCD0LOPT.0 size=1048576 2024-05-09T20:16:55.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~GIGLBCD0LOPT.1 size=673778 2024-05-09T20:16:55.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.424+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=GIGLBCD0LOPT tag=JZH5AWUAXKK1 type=write size=2770930 2024-05-09T20:16:55.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.428+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=FIIIWCU0IIF3 type=del 2024-05-09T20:16:55.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.432+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=ZYA7JKHAQVRE type=del size=3320735 2024-05-09T20:16:55.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.436+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~X4AX63N193Z4.0 size=1048576 2024-05-09T20:16:55.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.436+0000 7fdc882e58c0 1 < canceled part key=_multipart_CNV7NHP27JZ7.2~X4AX63N193Z4.1 size=1048576 2024-05-09T20:16:55.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.436+0000 7fdc882e58c0 1 < canceled part key=_multipart_CNV7NHP27JZ7.2~X4AX63N193Z4.2 size=120416 2024-05-09T20:16:55.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.440+0000 7fdc882e58c0 1 < canceled multipart upload key=CNV7NHP27JZ7 upload=X4AX63N193Z4 tag=NPASWL4A0AD8 type=write size=3266144 2024-05-09T20:16:55.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.444+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=Z30TQVQ68JXW type=del size=1699109 2024-05-09T20:16:55.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.452+0000 7fdc882e58c0 1 < suggested remove operation key=CNV7NHP27JZ7 tag=U700W4RIISEI type=del 2024-05-09T20:16:55.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.456+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=Z6TCJDXEZKVZ type=del size=3447816 2024-05-09T20:16:55.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.456+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=1O8U0KAYEQZA type=del 2024-05-09T20:16:55.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.460+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=Q7IEDBTCFKM4 type=write size=472014 2024-05-09T20:16:55.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.464+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=B6W3BZ2BCXZ7 type=write size=305893 2024-05-09T20:16:55.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.472+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~0ATH09DI2RV6.0 size=1048576 2024-05-09T20:16:55.465 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.472+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~0ATH09DI2RV6.1 size=6639 2024-05-09T20:16:55.465 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.472+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=0ATH09DI2RV6 tag=VN5K1M7SUQDJ type=write size=2103791 2024-05-09T20:16:55.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.476+0000 7fdc882e58c0 1 < completed write operation key=NYHR6GTPQYZS tag=3I3JC703787L type=write size=1027410 2024-05-09T20:16:55.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~WAHLYD08FDT3.0 size=687078 2024-05-09T20:16:55.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.484+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=WAHLYD08FDT3 tag=P27B777PN9WS type=write size=1735654 2024-05-09T20:16:55.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.488+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~IH06PIAIHGUK.0 size=1048576 2024-05-09T20:16:55.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.492+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~IH06PIAIHGUK.1 size=986516 2024-05-09T20:16:55.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.496+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=IH06PIAIHGUK tag=OYHWFZFGGUM2 type=write size=3083668 2024-05-09T20:16:55.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.504+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~57J3S4FLINOS.0 size=1048576 2024-05-09T20:16:55.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.508+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~57J3S4FLINOS.1 size=1048576 2024-05-09T20:16:55.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.508+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~57J3S4FLINOS.2 size=507238 2024-05-09T20:16:55.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.508+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=57J3S4FLINOS tag=UWBQ17556489 type=write size=3652966 2024-05-09T20:16:55.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.512+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~TT8SBE6G2JWX.0 size=1048576 2024-05-09T20:16:55.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.516+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~TT8SBE6G2JWX.1 size=1048576 2024-05-09T20:16:55.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.520+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~TT8SBE6G2JWX.2 size=650565 2024-05-09T20:16:55.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.524+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=TT8SBE6G2JWX tag=44K4KS8L6BJ0 type=write size=3796293 2024-05-09T20:16:55.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.528+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=PG5WC1S8TRC6 type=del size=2013444 2024-05-09T20:16:55.523 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.532+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~7RIWWR4HZHZ3.0 size=739998 2024-05-09T20:16:55.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.536+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=7RIWWR4HZHZ3 tag=19WSKADXC92D type=write size=1788574 2024-05-09T20:16:55.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.540+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=WATEFNZNQHMD type=del size=2389596 2024-05-09T20:16:55.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.544+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=W1Z1CC9H6V8I type=del 2024-05-09T20:16:55.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.548+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~79J1U0KA8QG7.0 size=1048576 2024-05-09T20:16:55.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.552+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~79J1U0KA8QG7.1 size=1048576 2024-05-09T20:16:55.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.556+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~79J1U0KA8QG7.2 size=847136 2024-05-09T20:16:55.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.556+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=79J1U0KA8QG7 tag=5Y871Q318B7E type=write size=3992864 2024-05-09T20:16:55.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.560+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=LZPXLNLYSU53 type=del 2024-05-09T20:16:55.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.564+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=9E45JONQIBJ8 type=del size=2986761 2024-05-09T20:16:55.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.564+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~TBJENM8SASDN.0 size=1048576 2024-05-09T20:16:55.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.568+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~TBJENM8SASDN.1 size=1048576 2024-05-09T20:16:55.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.572+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~TBJENM8SASDN.2 size=500873 2024-05-09T20:16:55.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.576+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=TBJENM8SASDN tag=BHB6XXYLTWYK type=write size=3646601 2024-05-09T20:16:55.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.584+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~E2GASXPV7TRW.0 size=1048576 2024-05-09T20:16:55.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~E2GASXPV7TRW.1 size=1048576 2024-05-09T20:16:55.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~E2GASXPV7TRW.2 size=530627 2024-05-09T20:16:55.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.588+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=E2GASXPV7TRW tag=H5YOPVCARO9Z type=write size=3676355 2024-05-09T20:16:55.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.592+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=WW7YM4TBWYI7 type=del 2024-05-09T20:16:55.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.596+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=0ST0HKK3CZOS type=write size=805303 2024-05-09T20:16:55.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.600+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=U0WN3N66ZYHP type=del 2024-05-09T20:16:55.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.608+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~TJF3IUOHNZN0.0 size=1048576 2024-05-09T20:16:55.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.608+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~TJF3IUOHNZN0.1 size=3404 2024-05-09T20:16:55.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.608+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=TJF3IUOHNZN0 tag=5SVS3Y1XUMZD type=write size=2100556 2024-05-09T20:16:55.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.612+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~AMH0KMXJK7XJ.0 size=1048576 2024-05-09T20:16:55.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.616+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~AMH0KMXJK7XJ.1 size=593639 2024-05-09T20:16:55.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.620+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=AMH0KMXJK7XJ tag=F9J446AQ6SHK type=write size=2690791 2024-05-09T20:16:55.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.624+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=6U16GK41X5F9 type=write size=589807 2024-05-09T20:16:55.621 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.628+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~GIGLBCD0LOPT.0 size=1048576 2024-05-09T20:16:55.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.632+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~GIGLBCD0LOPT.1 size=673778 2024-05-09T20:16:55.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.636+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=GIGLBCD0LOPT tag=JZH5AWUAXKK1 type=write size=2770930 2024-05-09T20:16:55.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.640+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=QRXIYUSHD7K2 type=del size=3654388 2024-05-09T20:16:55.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.644+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=ZYA7JKHAQVRE type=del 2024-05-09T20:16:55.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.652+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~M1B9FJJEEJJ9.0 size=1048576 2024-05-09T20:16:55.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.656+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~M1B9FJJEEJJ9.1 size=570271 2024-05-09T20:16:55.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.656+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=M1B9FJJEEJJ9 tag=1642I2268LVB type=write size=2667423 2024-05-09T20:16:55.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.660+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=Z30TQVQ68JXW type=del 2024-05-09T20:16:55.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.664+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=YXZOLC2OARE5 type=del size=875365 2024-05-09T20:16:55.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.668+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=Z6TCJDXEZKVZ type=del 2024-05-09T20:16:55.667 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.676+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~G5O8SNWGVIYS.0 size=1048576 2024-05-09T20:16:55.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.676+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~G5O8SNWGVIYS.1 size=1048576 2024-05-09T20:16:55.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.680+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~G5O8SNWGVIYS.2 size=504131 2024-05-09T20:16:55.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.680+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=G5O8SNWGVIYS tag=03RPQKFRRH6R type=write size=3649859 2024-05-09T20:16:55.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.684+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=Q7IEDBTCFKM4 type=write size=472014 2024-05-09T20:16:55.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.688+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=OI7QFAF2HSEN type=del size=2314642 2024-05-09T20:16:55.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.692+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~0ATH09DI2RV6.0 size=1048576 2024-05-09T20:16:55.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.696+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~0ATH09DI2RV6.1 size=6639 2024-05-09T20:16:55.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.700+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=0ATH09DI2RV6 tag=VN5K1M7SUQDJ type=write size=2103791 2024-05-09T20:16:55.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.708+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~53SVHAABUXFB.0 size=1048576 2024-05-09T20:16:55.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~53SVHAABUXFB.1 size=1048576 2024-05-09T20:16:55.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~53SVHAABUXFB.2 size=286984 2024-05-09T20:16:55.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.712+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=53SVHAABUXFB tag=OGSQ5477Y0T8 type=write size=3432712 2024-05-09T20:16:55.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.716+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~WAHLYD08FDT3.0 size=687078 2024-05-09T20:16:55.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.720+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=WAHLYD08FDT3 tag=P27B777PN9WS type=write size=1735654 2024-05-09T20:16:55.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.728+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~F109A4Y1T7HF.0 size=258257 2024-05-09T20:16:55.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.728+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=F109A4Y1T7HF tag=KDPID9G5MF2K type=write size=1306833 2024-05-09T20:16:55.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.732+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~57J3S4FLINOS.0 size=1048576 2024-05-09T20:16:55.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.732+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~57J3S4FLINOS.1 size=1048576 2024-05-09T20:16:55.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.736+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~57J3S4FLINOS.2 size=507238 2024-05-09T20:16:55.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.740+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=57J3S4FLINOS tag=UWBQ17556489 type=write size=3652966 2024-05-09T20:16:55.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.744+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=IIB0RKW289FZ type=del size=3428556 2024-05-09T20:16:55.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.748+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=PG5WC1S8TRC6 type=del 2024-05-09T20:16:55.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.752+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=02OWM5MO2S34 type=del size=1061097 2024-05-09T20:16:55.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.752+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=WATEFNZNQHMD type=del 2024-05-09T20:16:55.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.760+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~M50TZ47JIIIM.0 size=1048576 2024-05-09T20:16:55.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.760+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~M50TZ47JIIIM.1 size=428122 2024-05-09T20:16:55.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.760+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=M50TZ47JIIIM tag=5J1VK7PKXHPC type=write size=2525274 2024-05-09T20:16:55.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.764+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~79J1U0KA8QG7.0 size=1048576 2024-05-09T20:16:55.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.768+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~79J1U0KA8QG7.1 size=1048576 2024-05-09T20:16:55.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.768+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~79J1U0KA8QG7.2 size=847136 2024-05-09T20:16:55.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.776+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=79J1U0KA8QG7 tag=5Y871Q318B7E type=write size=3992864 2024-05-09T20:16:55.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.780+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=1LFZU9F4SQIC type=del size=2349253 2024-05-09T20:16:55.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.784+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=9E45JONQIBJ8 type=del 2024-05-09T20:16:55.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.788+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~LHIR11GEW3GK.0 size=775136 2024-05-09T20:16:55.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.788+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=LHIR11GEW3GK tag=D7FWNO9KO81I type=write size=1823712 2024-05-09T20:16:55.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.792+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~E2GASXPV7TRW.0 size=1048576 2024-05-09T20:16:55.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.796+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~E2GASXPV7TRW.1 size=1048576 2024-05-09T20:16:55.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.800+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~E2GASXPV7TRW.2 size=530627 2024-05-09T20:16:55.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.804+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=E2GASXPV7TRW tag=H5YOPVCARO9Z type=write size=3676355 2024-05-09T20:16:55.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.808+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=RZAOD5AN4PN4 type=del size=1030762 2024-05-09T20:16:55.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.812+0000 7fdc882e58c0 1 < suggested remove operation key=OGVA2778MJWU tag=0ST0HKK3CZOS type=write 2024-05-09T20:16:55.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.820+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~NGBHIG83OAAH.0 size=1048576 2024-05-09T20:16:55.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.820+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~NGBHIG83OAAH.1 size=599644 2024-05-09T20:16:55.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.820+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=NGBHIG83OAAH tag=IOBGK9D0LXJV type=write size=2696796 2024-05-09T20:16:55.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.824+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~TJF3IUOHNZN0.0 size=1048576 2024-05-09T20:16:55.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.828+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~TJF3IUOHNZN0.1 size=3404 2024-05-09T20:16:55.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.832+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=TJF3IUOHNZN0 tag=5SVS3Y1XUMZD type=write size=2100556 2024-05-09T20:16:55.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.832+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=3O5BIJHMPDZP type=del size=3292371 2024-05-09T20:16:55.828 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.836+0000 7fdc882e58c0 1 < canceled operation key=HPK2T0629B1P tag=6U16GK41X5F9 type=write size=589807 2024-05-09T20:16:55.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.840+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~KLLOS6G07NFY.0 size=1048576 2024-05-09T20:16:55.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.844+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~KLLOS6G07NFY.1 size=957778 2024-05-09T20:16:55.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.844+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=KLLOS6G07NFY tag=W4INB4RRJQOL type=write size=3054930 2024-05-09T20:16:55.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.844+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=QRXIYUSHD7K2 type=del 2024-05-09T20:16:55.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.848+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=UXGLD8F6JEJZ type=write size=889982 2024-05-09T20:16:55.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.852+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~M1B9FJJEEJJ9.0 size=1048576 2024-05-09T20:16:55.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.852+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~M1B9FJJEEJJ9.1 size=570271 2024-05-09T20:16:55.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.860+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=M1B9FJJEEJJ9 tag=1642I2268LVB type=write size=2667423 2024-05-09T20:16:55.853 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.860+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=1ICQWD2VW39E type=del size=3279053 2024-05-09T20:16:55.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.864+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=YXZOLC2OARE5 type=del 2024-05-09T20:16:55.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.868+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=NS8LM74Y9IN7 type=del size=2046696 2024-05-09T20:16:55.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.872+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~G5O8SNWGVIYS.0 size=1048576 2024-05-09T20:16:55.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.872+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~G5O8SNWGVIYS.1 size=1048576 2024-05-09T20:16:55.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.876+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~G5O8SNWGVIYS.2 size=504131 2024-05-09T20:16:55.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.880+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=G5O8SNWGVIYS tag=03RPQKFRRH6R type=write size=3649859 2024-05-09T20:16:55.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.884+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=4X6HR9P11R6D type=del size=3729936 2024-05-09T20:16:55.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.888+0000 7fdc882e58c0 1 < canceled operation key=5H2GWM9LQELZ tag=OI7QFAF2HSEN type=del size=2314642 2024-05-09T20:16:55.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.892+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~M5K2R4WQ7UB6.0 size=908759 2024-05-09T20:16:55.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.892+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=M5K2R4WQ7UB6 tag=98LXYK3N8QTW type=write size=1957335 2024-05-09T20:16:55.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.896+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~53SVHAABUXFB.0 size=1048576 2024-05-09T20:16:55.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.900+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~53SVHAABUXFB.1 size=1048576 2024-05-09T20:16:55.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.900+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~53SVHAABUXFB.2 size=286984 2024-05-09T20:16:55.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.904+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=53SVHAABUXFB tag=OGSQ5477Y0T8 type=write size=3432712 2024-05-09T20:16:55.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.908+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=3SHIKRZN7KGU type=del size=706723 2024-05-09T20:16:55.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.912+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~F109A4Y1T7HF.0 size=258257 2024-05-09T20:16:55.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.916+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=F109A4Y1T7HF tag=KDPID9G5MF2K type=write size=1306833 2024-05-09T20:16:55.911 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.920+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=FPYLP90T9O4G type=del size=958921 2024-05-09T20:16:55.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.924+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=IIB0RKW289FZ type=del 2024-05-09T20:16:55.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.928+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=Q407GXFLH83J type=write size=456903 2024-05-09T20:16:55.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.932+0000 7fdc882e58c0 1 < suggested remove operation key=CNV7NHP27JZ7 tag=02OWM5MO2S34 type=del 2024-05-09T20:16:55.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~WJ2Z62QSN33O.0 size=1048576 2024-05-09T20:16:55.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.940+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~WJ2Z62QSN33O.1 size=1048576 2024-05-09T20:16:55.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.944+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~WJ2Z62QSN33O.2 size=432958 2024-05-09T20:16:55.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.944+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=WJ2Z62QSN33O tag=1R3NOQ66I1LG type=write size=3578686 2024-05-09T20:16:55.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.948+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~M50TZ47JIIIM.0 size=1048576 2024-05-09T20:16:55.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.948+0000 7fdc882e58c0 1 < canceled part key=_multipart_O07UHAIJZF0R.2~M50TZ47JIIIM.1 size=428122 2024-05-09T20:16:55.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.956+0000 7fdc882e58c0 1 < canceled multipart upload key=O07UHAIJZF0R upload=M50TZ47JIIIM tag=5J1VK7PKXHPC type=write size=2525274 2024-05-09T20:16:55.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.960+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=CK3GLTOOLMOB type=del size=1226868 2024-05-09T20:16:55.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.960+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=1LFZU9F4SQIC type=del 2024-05-09T20:16:55.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.968+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~R8IPBH4FIAZN.0 size=1048576 2024-05-09T20:16:55.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.972+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~R8IPBH4FIAZN.1 size=561643 2024-05-09T20:16:55.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.972+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=R8IPBH4FIAZN tag=WHW1WL73D4DZ type=write size=2658795 2024-05-09T20:16:55.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.972+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~LHIR11GEW3GK.0 size=775136 2024-05-09T20:16:55.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.980+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=LHIR11GEW3GK tag=D7FWNO9KO81I type=write size=1823712 2024-05-09T20:16:55.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.984+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=DZLV4UBAX73E type=write size=38119 2024-05-09T20:16:55.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.988+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=RZAOD5AN4PN4 type=del 2024-05-09T20:16:55.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.988+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=NMECMO0SA2AA type=del size=1614269 2024-05-09T20:16:55.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.992+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~NGBHIG83OAAH.0 size=1048576 2024-05-09T20:16:55.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:55.996+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~NGBHIG83OAAH.1 size=599644 2024-05-09T20:16:55.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.004+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=NGBHIG83OAAH tag=IOBGK9D0LXJV type=write size=2696796 2024-05-09T20:16:55.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.004+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=7WNQJ8NB6E6R type=del size=1201976 2024-05-09T20:16:56.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.008+0000 7fdc882e58c0 1 < canceled operation key=P2DNA4DGI54P tag=3O5BIJHMPDZP type=del size=3292371 2024-05-09T20:16:56.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~44HE7OIL5O02.0 size=1048576 2024-05-09T20:16:56.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~44HE7OIL5O02.1 size=750018 2024-05-09T20:16:56.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.016+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=44HE7OIL5O02 tag=GO01TTKEMQKV type=write size=2847170 2024-05-09T20:16:56.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.020+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~KLLOS6G07NFY.0 size=1048576 2024-05-09T20:16:56.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.020+0000 7fdc882e58c0 1 < canceled part key=_multipart_QM53IENH9CX4.2~KLLOS6G07NFY.1 size=957778 2024-05-09T20:16:56.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.028+0000 7fdc882e58c0 1 < canceled multipart upload key=QM53IENH9CX4 upload=KLLOS6G07NFY tag=W4INB4RRJQOL type=write size=3054930 2024-05-09T20:16:56.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.032+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=IZ5E8J6SQX2A type=del size=278318 2024-05-09T20:16:56.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.032+0000 7fdc882e58c0 1 < canceled operation key=5UMJFJRKUPKR tag=UXGLD8F6JEJZ type=write size=889982 2024-05-09T20:16:56.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.036+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=YUWWNAEE4EAT type=del size=1976463 2024-05-09T20:16:56.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.040+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=1ICQWD2VW39E type=del 2024-05-09T20:16:56.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.044+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=Z2W5A75XJ6JB type=del size=2125256 2024-05-09T20:16:56.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.048+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=NS8LM74Y9IN7 type=del 2024-05-09T20:16:56.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.052+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=654RG4QE4NTL type=del size=359547 2024-05-09T20:16:56.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.056+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=4X6HR9P11R6D type=del 2024-05-09T20:16:56.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.060+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=U3TU83AQ0NJH type=del size=832110 2024-05-09T20:16:56.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.060+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~M5K2R4WQ7UB6.0 size=908759 2024-05-09T20:16:56.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.068+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=M5K2R4WQ7UB6 tag=98LXYK3N8QTW type=write size=1957335 2024-05-09T20:16:56.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.072+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=UYS99JYT2KGX type=del size=1494966 2024-05-09T20:16:56.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.076+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=3SHIKRZN7KGU type=del 2024-05-09T20:16:56.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.080+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~H2BWOITAP3Y6.0 size=1048576 2024-05-09T20:16:56.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.084+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~H2BWOITAP3Y6.1 size=1048576 2024-05-09T20:16:56.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.088+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~H2BWOITAP3Y6.2 size=519577 2024-05-09T20:16:56.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.088+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=H2BWOITAP3Y6 tag=2Q0LPKTHF7NN type=write size=3665305 2024-05-09T20:16:56.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.088+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=FPYLP90T9O4G type=del 2024-05-09T20:16:56.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.092+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=MHA27Y2VXRMG type=del size=1942461 2024-05-09T20:16:56.088 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.096+0000 7fdc882e58c0 1 < completed write operation key=SPXHCWZICCWY tag=Q407GXFLH83J type=write size=456903 2024-05-09T20:16:56.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.104+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~C4ILNZZXDXD8.0 size=1048576 2024-05-09T20:16:56.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.104+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~C4ILNZZXDXD8.1 size=46529 2024-05-09T20:16:56.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.104+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=C4ILNZZXDXD8 tag=3124IJ5UHCMO type=write size=2143681 2024-05-09T20:16:56.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.108+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~WJ2Z62QSN33O.0 size=1048576 2024-05-09T20:16:56.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.112+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~WJ2Z62QSN33O.1 size=1048576 2024-05-09T20:16:56.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.116+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~WJ2Z62QSN33O.2 size=432958 2024-05-09T20:16:56.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.120+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=WJ2Z62QSN33O tag=1R3NOQ66I1LG type=write size=3578686 2024-05-09T20:16:56.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.124+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~YSF8WBXAPJHG.0 size=370792 2024-05-09T20:16:56.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.124+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=YSF8WBXAPJHG tag=ZPGNXD0RMJ7S type=write size=1419368 2024-05-09T20:16:56.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.128+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=CK3GLTOOLMOB type=del 2024-05-09T20:16:56.124 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.132+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=KKTWD443NQIR type=del size=3488865 2024-05-09T20:16:56.127 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.136+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~R8IPBH4FIAZN.0 size=1048576 2024-05-09T20:16:56.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.136+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~R8IPBH4FIAZN.1 size=561643 2024-05-09T20:16:56.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.144+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=R8IPBH4FIAZN tag=WHW1WL73D4DZ type=write size=2658795 2024-05-09T20:16:56.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.148+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=G0NBKK9TPC9R type=del size=1985461 2024-05-09T20:16:56.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.148+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=DZLV4UBAX73E type=write size=38119 2024-05-09T20:16:56.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.156+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~SUUH798V8FB8.0 size=247871 2024-05-09T20:16:56.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.156+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=SUUH798V8FB8 tag=8ARZTGA9JTF2 type=write size=1296447 2024-05-09T20:16:56.149 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.156+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=NMECMO0SA2AA type=del 2024-05-09T20:16:56.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.164+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~3T1GOKXSNMZY.0 size=1048576 2024-05-09T20:16:56.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.168+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~3T1GOKXSNMZY.1 size=1048576 2024-05-09T20:16:56.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.168+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~3T1GOKXSNMZY.2 size=667376 2024-05-09T20:16:56.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.168+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=3T1GOKXSNMZY tag=15DTQV6JE2J7 type=write size=3813104 2024-05-09T20:16:56.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.172+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=7WNQJ8NB6E6R type=del 2024-05-09T20:16:56.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.176+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=P8XDUBK3W56P type=del size=1864970 2024-05-09T20:16:56.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.180+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~44HE7OIL5O02.0 size=1048576 2024-05-09T20:16:56.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.184+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~44HE7OIL5O02.1 size=750018 2024-05-09T20:16:56.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.188+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=44HE7OIL5O02 tag=GO01TTKEMQKV type=write size=2847170 2024-05-09T20:16:56.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.196+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~WG0HU3CGUK46.0 size=1048576 2024-05-09T20:16:56.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.200+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~WG0HU3CGUK46.1 size=386776 2024-05-09T20:16:56.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.200+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=WG0HU3CGUK46 tag=7UPAIHY40E4G type=write size=2483928 2024-05-09T20:16:56.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.204+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=IZ5E8J6SQX2A type=del 2024-05-09T20:16:56.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.208+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=06N62RTKN0UR type=del size=246307 2024-05-09T20:16:56.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.212+0000 7fdc882e58c0 1 < canceled operation key=G0DX1F257RLD tag=YUWWNAEE4EAT type=del size=1976463 2024-05-09T20:16:56.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.216+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~5K8U2NHVQ52B.0 size=1048576 2024-05-09T20:16:56.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.220+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~5K8U2NHVQ52B.1 size=607791 2024-05-09T20:16:56.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.220+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=5K8U2NHVQ52B tag=2B1UO886WHHO type=write size=2704943 2024-05-09T20:16:56.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.224+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=Z2W5A75XJ6JB type=del 2024-05-09T20:16:56.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.228+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=6X7HJ0ND2EKE type=write size=908610 2024-05-09T20:16:56.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.232+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=654RG4QE4NTL type=del 2024-05-09T20:16:56.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.236+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=ARLQ2MDKA2UJ type=write size=420227 2024-05-09T20:16:56.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.240+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=U3TU83AQ0NJH type=del 2024-05-09T20:16:56.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.244+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=1WWUB6PPNTBU type=del size=785435 2024-05-09T20:16:56.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.252+0000 7fdc882e58c0 1 < suggested update operation key=IZPPFLWVPRKO tag=UYS99JYT2KGX type=del size=1494966 2024-05-09T20:16:56.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.252+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=8NY3KSGCLNAB type=write size=972011 2024-05-09T20:16:56.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.256+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~H2BWOITAP3Y6.0 size=1048576 2024-05-09T20:16:56.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.260+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~H2BWOITAP3Y6.1 size=1048576 2024-05-09T20:16:56.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.264+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~H2BWOITAP3Y6.2 size=519577 2024-05-09T20:16:56.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.272+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=H2BWOITAP3Y6 tag=2Q0LPKTHF7NN type=write size=3665305 2024-05-09T20:16:56.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.276+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=NLMI6H6ODQN4 type=del size=683776 2024-05-09T20:16:56.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.280+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=MHA27Y2VXRMG type=del 2024-05-09T20:16:56.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.284+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=521R69GTGL5H type=del size=2933675 2024-05-09T20:16:56.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.288+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~C4ILNZZXDXD8.0 size=1048576 2024-05-09T20:16:56.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.292+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~C4ILNZZXDXD8.1 size=46529 2024-05-09T20:16:56.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.300+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=C4ILNZZXDXD8 tag=3124IJ5UHCMO type=write size=2143681 2024-05-09T20:16:56.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.304+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=04ZQHNA5Y4IY type=del size=2671063 2024-05-09T20:16:56.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.308+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~YSF8WBXAPJHG.0 size=370792 2024-05-09T20:16:56.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.312+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=YSF8WBXAPJHG tag=ZPGNXD0RMJ7S type=write size=1419368 2024-05-09T20:16:56.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.320+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~CSNUNKHITV3Y.0 size=1048576 2024-05-09T20:16:56.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.320+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~CSNUNKHITV3Y.1 size=602331 2024-05-09T20:16:56.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.320+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=CSNUNKHITV3Y tag=JZCK6Y6BCOMF type=write size=2699483 2024-05-09T20:16:56.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.324+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=KKTWD443NQIR type=del 2024-05-09T20:16:56.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.328+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=UGEADW6D8832 type=del size=2915189 2024-05-09T20:16:56.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.332+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=G0NBKK9TPC9R type=del 2024-05-09T20:16:56.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.336+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=U7P5K6OXYLWY type=del size=1704173 2024-05-09T20:16:56.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.340+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~SUUH798V8FB8.0 size=247871 2024-05-09T20:16:56.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.348+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=SUUH798V8FB8 tag=8ARZTGA9JTF2 type=write size=1296447 2024-05-09T20:16:56.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.352+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=7KNZNT912R0U type=write size=853548 2024-05-09T20:16:56.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.356+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~3T1GOKXSNMZY.0 size=1048576 2024-05-09T20:16:56.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.356+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~3T1GOKXSNMZY.1 size=1048576 2024-05-09T20:16:56.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.360+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~3T1GOKXSNMZY.2 size=667376 2024-05-09T20:16:56.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.368+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=3T1GOKXSNMZY tag=15DTQV6JE2J7 type=write size=3813104 2024-05-09T20:16:56.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.372+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~E0B1KGU0Y2E7.0 size=1048576 2024-05-09T20:16:56.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.376+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~E0B1KGU0Y2E7.1 size=709399 2024-05-09T20:16:56.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.376+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=E0B1KGU0Y2E7 tag=JRXN8W3MPBK9 type=write size=2806551 2024-05-09T20:16:56.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.384+0000 7fdc882e58c0 1 < suggested update operation key=O07UHAIJZF0R tag=P8XDUBK3W56P type=del size=1864970 2024-05-09T20:16:56.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.392+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~DDXJP5PVIYZD.0 size=738998 2024-05-09T20:16:56.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.392+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=DDXJP5PVIYZD tag=BIX6BYPGGRIE type=write size=1787574 2024-05-09T20:16:56.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.396+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~WG0HU3CGUK46.0 size=1048576 2024-05-09T20:16:56.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.396+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~WG0HU3CGUK46.1 size=386776 2024-05-09T20:16:56.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.404+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=WG0HU3CGUK46 tag=7UPAIHY40E4G type=write size=2483928 2024-05-09T20:16:56.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.408+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~41YIE7D3NLXY.0 size=1048576 2024-05-09T20:16:56.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.412+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~41YIE7D3NLXY.1 size=1048576 2024-05-09T20:16:56.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.416+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~41YIE7D3NLXY.2 size=937806 2024-05-09T20:16:56.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.416+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=41YIE7D3NLXY tag=Y6N2YM8MYPVJ type=write size=4083534 2024-05-09T20:16:56.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.424+0000 7fdc882e58c0 1 < suggested update operation key=NBAECV9L85UK tag=06N62RTKN0UR type=del size=246307 2024-05-09T20:16:56.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.424+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=3Z3LKL9WIWRV type=del size=3608630 2024-05-09T20:16:56.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.428+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~5K8U2NHVQ52B.0 size=1048576 2024-05-09T20:16:56.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.432+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~5K8U2NHVQ52B.1 size=607791 2024-05-09T20:16:56.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.436+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=5K8U2NHVQ52B tag=2B1UO886WHHO type=write size=2704943 2024-05-09T20:16:56.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.440+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=36JYXC8Z9YIG type=del size=4010113 2024-05-09T20:16:56.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.444+0000 7fdc882e58c0 1 < completed write operation key=HPK2T0629B1P tag=6X7HJ0ND2EKE type=write size=908610 2024-05-09T20:16:56.443 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.452+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~0RE8BGO02VFL.0 size=1048576 2024-05-09T20:16:56.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.452+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~0RE8BGO02VFL.1 size=1048576 2024-05-09T20:16:56.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.456+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~0RE8BGO02VFL.2 size=98345 2024-05-09T20:16:56.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.456+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=0RE8BGO02VFL tag=8E5DAJT87IRV type=write size=3244073 2024-05-09T20:16:56.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.460+0000 7fdc882e58c0 -1 < failed to complete operation key=O07UHAIJZF0R tag=ARLQ2MDKA2UJ type=write size=420227: (22) Invalid argument 2024-05-09T20:16:56.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.464+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=P671KP2E9XHU type=del size=2578866 2024-05-09T20:16:56.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.468+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=1WWUB6PPNTBU type=del 2024-05-09T20:16:56.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.472+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=L7P8O1L6TCW4 type=del size=1705150 2024-05-09T20:16:56.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.472+0000 7fdc882e58c0 1 < completed write operation key=WC5YH0OU51WX tag=8NY3KSGCLNAB type=write size=972011 2024-05-09T20:16:56.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.476+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=YTE5GS5JLVJB type=del size=356036 2024-05-09T20:16:56.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.480+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=NLMI6H6ODQN4 type=del 2024-05-09T20:16:56.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.484+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=R522178IBOJH type=del size=2547133 2024-05-09T20:16:56.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.488+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=521R69GTGL5H type=del 2024-05-09T20:16:56.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.492+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=N6X8D38MQ9N3 type=write size=517380 2024-05-09T20:16:56.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.500+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=04ZQHNA5Y4IY type=del size=2671063 2024-05-09T20:16:56.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.504+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=33Z81RCFO67N type=del size=2662555 2024-05-09T20:16:56.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.508+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~CSNUNKHITV3Y.0 size=1048576 2024-05-09T20:16:56.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.508+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~CSNUNKHITV3Y.1 size=602331 2024-05-09T20:16:56.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.516+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=CSNUNKHITV3Y tag=JZCK6Y6BCOMF type=write size=2699483 2024-05-09T20:16:56.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.524+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~5LQ306TYTEQO.0 size=1048576 2024-05-09T20:16:56.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.528+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~5LQ306TYTEQO.1 size=517810 2024-05-09T20:16:56.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.528+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=5LQ306TYTEQO tag=Q6S06KAMWFY5 type=write size=2614962 2024-05-09T20:16:56.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.528+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=UGEADW6D8832 type=del 2024-05-09T20:16:56.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.536+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~ZAXFOHGOT1AW.0 size=372325 2024-05-09T20:16:56.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.536+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=ZAXFOHGOT1AW tag=V3E61GGN9KEJ type=write size=1420901 2024-05-09T20:16:56.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.540+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=U7P5K6OXYLWY type=del 2024-05-09T20:16:56.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.544+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=V5XJVW6IWUJR type=del size=1227900 2024-05-09T20:16:56.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.548+0000 7fdc882e58c0 1 < completed write operation key=8FFZZDURUWM1 tag=7KNZNT912R0U type=write size=853548 2024-05-09T20:16:56.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.552+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~CO0WSJ6G0C00.0 size=1048576 2024-05-09T20:16:56.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.556+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~CO0WSJ6G0C00.1 size=580469 2024-05-09T20:16:56.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.556+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=CO0WSJ6G0C00 tag=GJ2BBZ49XUOM type=write size=2677621 2024-05-09T20:16:56.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.560+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~E0B1KGU0Y2E7.0 size=1048576 2024-05-09T20:16:56.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.560+0000 7fdc882e58c0 1 < canceled part key=_multipart_OGVA2778MJWU.2~E0B1KGU0Y2E7.1 size=709399 2024-05-09T20:16:56.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.564+0000 7fdc882e58c0 1 < canceled multipart upload key=OGVA2778MJWU upload=E0B1KGU0Y2E7 tag=JRXN8W3MPBK9 type=write size=2806551 2024-05-09T20:16:56.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.568+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=M1ODZ8UVCC6C type=del size=134580 2024-05-09T20:16:56.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.572+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~DDXJP5PVIYZD.0 size=738998 2024-05-09T20:16:56.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.580+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=DDXJP5PVIYZD tag=BIX6BYPGGRIE type=write size=1787574 2024-05-09T20:16:56.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.584+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~XADKAL38ZHT0.0 size=1048576 2024-05-09T20:16:56.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~XADKAL38ZHT0.1 size=563883 2024-05-09T20:16:56.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.588+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=XADKAL38ZHT0 tag=KPOE712W47F0 type=write size=2661035 2024-05-09T20:16:56.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.592+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~41YIE7D3NLXY.0 size=1048576 2024-05-09T20:16:56.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.596+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~41YIE7D3NLXY.1 size=1048576 2024-05-09T20:16:56.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.596+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~41YIE7D3NLXY.2 size=937806 2024-05-09T20:16:56.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.604+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=41YIE7D3NLXY tag=Y6N2YM8MYPVJ type=write size=4083534 2024-05-09T20:16:56.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~010TI3QB1WZQ.0 size=1048576 2024-05-09T20:16:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.616+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~010TI3QB1WZQ.1 size=1048576 2024-05-09T20:16:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.616+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~010TI3QB1WZQ.2 size=699095 2024-05-09T20:16:56.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.616+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=010TI3QB1WZQ tag=K54NGQE9W0F0 type=write size=3844823 2024-05-09T20:16:56.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.620+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=3Z3LKL9WIWRV type=del 2024-05-09T20:16:56.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.628+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~1JAR5XZOYGCZ.0 size=1048576 2024-05-09T20:16:56.621 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.628+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~1JAR5XZOYGCZ.1 size=755592 2024-05-09T20:16:56.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.628+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=1JAR5XZOYGCZ tag=WFBWWE81ACH2 type=write size=2852744 2024-05-09T20:16:56.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.632+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=36JYXC8Z9YIG type=del 2024-05-09T20:16:56.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.640+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~MSEZQZNR5XX8.0 size=1048576 2024-05-09T20:16:56.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~MSEZQZNR5XX8.1 size=1028645 2024-05-09T20:16:56.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.644+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=MSEZQZNR5XX8 tag=L5Q93V2LOXYE type=write size=3125797 2024-05-09T20:16:56.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.648+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~0RE8BGO02VFL.0 size=1048576 2024-05-09T20:16:56.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.648+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~0RE8BGO02VFL.1 size=1048576 2024-05-09T20:16:56.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.652+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~0RE8BGO02VFL.2 size=98345 2024-05-09T20:16:56.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.656+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=0RE8BGO02VFL tag=8E5DAJT87IRV type=write size=3244073 2024-05-09T20:16:56.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.660+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=W1FPY39II0MC type=del size=1623176 2024-05-09T20:16:56.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.664+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=P671KP2E9XHU type=del 2024-05-09T20:16:56.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.672+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~SBPASI8677JO.0 size=1048576 2024-05-09T20:16:56.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.676+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~SBPASI8677JO.1 size=381209 2024-05-09T20:16:56.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.676+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=SBPASI8677JO tag=4873QHQQ09L4 type=write size=2478361 2024-05-09T20:16:56.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.680+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=L7P8O1L6TCW4 type=del 2024-05-09T20:16:56.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.684+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~U1RTCQNUON6Q.0 size=1048576 2024-05-09T20:16:56.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.688+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~U1RTCQNUON6Q.1 size=135195 2024-05-09T20:16:56.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.688+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=U1RTCQNUON6Q tag=T8SCT9VW1JEQ type=write size=2232347 2024-05-09T20:16:56.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.692+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=YTE5GS5JLVJB type=del 2024-05-09T20:16:56.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.696+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=AJE9ZQ0CTS9E type=del size=3160019 2024-05-09T20:16:56.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.696+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=R522178IBOJH type=del 2024-05-09T20:16:56.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.704+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~A3WV725JQTJ0.0 size=1048576 2024-05-09T20:16:56.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.708+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~A3WV725JQTJ0.1 size=133652 2024-05-09T20:16:56.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.708+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=A3WV725JQTJ0 tag=IQL3EQ1AH4OX type=write size=2230804 2024-05-09T20:16:56.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.712+0000 7fdc882e58c0 1 < completed write operation key=QM53IENH9CX4 tag=N6X8D38MQ9N3 type=write size=517380 2024-05-09T20:16:56.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.716+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=PBIH7CDO5UJ2 type=del size=1748937 2024-05-09T20:16:56.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.720+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=33Z81RCFO67N type=del 2024-05-09T20:16:56.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.724+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=1NJO831IUDE1 type=del size=403017 2024-05-09T20:16:56.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.724+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~5LQ306TYTEQO.0 size=1048576 2024-05-09T20:16:56.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.728+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~5LQ306TYTEQO.1 size=517810 2024-05-09T20:16:56.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.736+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=5LQ306TYTEQO tag=Q6S06KAMWFY5 type=write size=2614962 2024-05-09T20:16:56.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.744+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~RLRYZZIEFON2.0 size=871070 2024-05-09T20:16:56.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.744+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=RLRYZZIEFON2 tag=I0K9OJ1BRN88 type=write size=1919646 2024-05-09T20:16:56.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.748+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~ZAXFOHGOT1AW.0 size=372325 2024-05-09T20:16:56.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.752+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=ZAXFOHGOT1AW tag=V3E61GGN9KEJ type=write size=1420901 2024-05-09T20:16:56.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.760+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~SL5P099W5RAD.0 size=1048576 2024-05-09T20:16:56.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.764+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~SL5P099W5RAD.1 size=1048576 2024-05-09T20:16:56.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.764+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~SL5P099W5RAD.2 size=388958 2024-05-09T20:16:56.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.764+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=SL5P099W5RAD tag=Z0FLKO5J3QFY type=write size=3534686 2024-05-09T20:16:56.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.772+0000 7fdc882e58c0 1 < suggested update operation key=5UMJFJRKUPKR tag=V5XJVW6IWUJR type=del size=1227900 2024-05-09T20:16:56.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~XFRHIWUTYKOJ.0 size=1048576 2024-05-09T20:16:56.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~XFRHIWUTYKOJ.1 size=1048576 2024-05-09T20:16:56.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~XFRHIWUTYKOJ.2 size=440508 2024-05-09T20:16:56.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.784+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=XFRHIWUTYKOJ tag=8OZU64E2T13L type=write size=3586236 2024-05-09T20:16:56.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.788+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~CO0WSJ6G0C00.0 size=1048576 2024-05-09T20:16:56.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.788+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~CO0WSJ6G0C00.1 size=580469 2024-05-09T20:16:56.786 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.796+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=CO0WSJ6G0C00 tag=GJ2BBZ49XUOM type=write size=2677621 2024-05-09T20:16:56.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.800+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~LAFOJ0SW6ZHE.0 size=925748 2024-05-09T20:16:56.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.800+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=LAFOJ0SW6ZHE tag=QLYTIQOY0TQ6 type=write size=1974324 2024-05-09T20:16:56.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.804+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=M1ODZ8UVCC6C type=del 2024-05-09T20:16:56.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.808+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=Y58ZF83N1N6A type=write size=261309 2024-05-09T20:16:56.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.808+0000 7fdc882e58c0 1 < canceled part key=_multipart_NNGWUSCETMH4.2~XADKAL38ZHT0.0 size=1048576 2024-05-09T20:16:56.802 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.812+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~XADKAL38ZHT0.1 size=563883 2024-05-09T20:16:56.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.816+0000 7fdc882e58c0 1 < canceled multipart upload key=NNGWUSCETMH4 upload=XADKAL38ZHT0 tag=KPOE712W47F0 type=write size=2661035 2024-05-09T20:16:56.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.820+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=VFKNWUNNF7WK type=del size=4101133 2024-05-09T20:16:56.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.824+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~010TI3QB1WZQ.0 size=1048576 2024-05-09T20:16:56.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.824+0000 7fdc882e58c0 1 < canceled part key=_multipart_WC5YH0OU51WX.2~010TI3QB1WZQ.1 size=1048576 2024-05-09T20:16:56.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.824+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~010TI3QB1WZQ.2 size=699095 2024-05-09T20:16:56.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.832+0000 7fdc882e58c0 1 < canceled multipart upload key=WC5YH0OU51WX upload=010TI3QB1WZQ tag=K54NGQE9W0F0 type=write size=3844823 2024-05-09T20:16:56.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.836+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=MKQXMFR00NIB type=del size=2855256 2024-05-09T20:16:56.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.836+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~1JAR5XZOYGCZ.0 size=1048576 2024-05-09T20:16:56.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.840+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~1JAR5XZOYGCZ.1 size=755592 2024-05-09T20:16:56.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.844+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=1JAR5XZOYGCZ tag=WFBWWE81ACH2 type=write size=2852744 2024-05-09T20:16:56.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.848+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=8IXTE2GN0FO9 type=del size=3029101 2024-05-09T20:16:56.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.852+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~MSEZQZNR5XX8.0 size=1048576 2024-05-09T20:16:56.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.856+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~MSEZQZNR5XX8.1 size=1028645 2024-05-09T20:16:56.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.860+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=MSEZQZNR5XX8 tag=L5Q93V2LOXYE type=write size=3125797 2024-05-09T20:16:56.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.864+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=WQEKIHSPCGSK type=del size=2367214 2024-05-09T20:16:56.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.868+0000 7fdc882e58c0 1 < suggested update operation key=TWWOL5D8C1MQ tag=W1FPY39II0MC type=del size=1623176 2024-05-09T20:16:56.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.876+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~39BQF4AG0TOS.0 size=1048576 2024-05-09T20:16:56.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.876+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~39BQF4AG0TOS.1 size=274358 2024-05-09T20:16:56.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.876+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=39BQF4AG0TOS tag=ZR6XJW8YS6U6 type=write size=2371510 2024-05-09T20:16:56.872 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.880+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~SBPASI8677JO.0 size=1048576 2024-05-09T20:16:56.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.884+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~SBPASI8677JO.1 size=381209 2024-05-09T20:16:56.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.888+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=SBPASI8677JO tag=4873QHQQ09L4 type=write size=2478361 2024-05-09T20:16:56.884 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.892+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=ICWQCR2OZIGP type=del size=1970498 2024-05-09T20:16:56.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.896+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~U1RTCQNUON6Q.0 size=1048576 2024-05-09T20:16:56.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.900+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~U1RTCQNUON6Q.1 size=135195 2024-05-09T20:16:56.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.904+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=U1RTCQNUON6Q tag=T8SCT9VW1JEQ type=write size=2232347 2024-05-09T20:16:56.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.908+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~2A8CJX15C6Q4.0 size=372457 2024-05-09T20:16:56.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.908+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=2A8CJX15C6Q4 tag=ZH5J9XQLJ8CU type=write size=1421033 2024-05-09T20:16:56.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.912+0000 7fdc882e58c0 1 < canceled operation key=5H2GWM9LQELZ tag=AJE9ZQ0CTS9E type=del size=3160019 2024-05-09T20:16:56.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.916+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=ZB77M1GIVL2E type=write size=798019 2024-05-09T20:16:56.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.920+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~A3WV725JQTJ0.0 size=1048576 2024-05-09T20:16:56.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.924+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~A3WV725JQTJ0.1 size=133652 2024-05-09T20:16:56.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.928+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=A3WV725JQTJ0 tag=IQL3EQ1AH4OX type=write size=2230804 2024-05-09T20:16:56.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.932+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=YBHLWV460TFA type=del size=2584653 2024-05-09T20:16:56.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.936+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=PBIH7CDO5UJ2 type=del 2024-05-09T20:16:56.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.940+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~K0JA3NCL5YJ0.0 size=1048576 2024-05-09T20:16:56.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.944+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~K0JA3NCL5YJ0.1 size=105174 2024-05-09T20:16:56.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.944+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=K0JA3NCL5YJ0 tag=1M01D59NRMFQ type=write size=2202326 2024-05-09T20:16:56.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.948+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=1NJO831IUDE1 type=del 2024-05-09T20:16:56.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.948+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=ILCRF77MPSKP type=del size=2703024 2024-05-09T20:16:56.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.952+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~RLRYZZIEFON2.0 size=871070 2024-05-09T20:16:56.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.956+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=RLRYZZIEFON2 tag=I0K9OJ1BRN88 type=write size=1919646 2024-05-09T20:16:56.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.960+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=YIQ9SUT7MIKI type=del size=2886113 2024-05-09T20:16:56.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.964+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~SL5P099W5RAD.0 size=1048576 2024-05-09T20:16:56.957 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.964+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~SL5P099W5RAD.1 size=1048576 2024-05-09T20:16:56.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.968+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~SL5P099W5RAD.2 size=388958 2024-05-09T20:16:56.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.972+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=SL5P099W5RAD tag=Z0FLKO5J3QFY type=write size=3534686 2024-05-09T20:16:56.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.976+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=16Q7OO8305LZ type=write size=123001 2024-05-09T20:16:56.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.980+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~XFRHIWUTYKOJ.0 size=1048576 2024-05-09T20:16:56.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.980+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~XFRHIWUTYKOJ.1 size=1048576 2024-05-09T20:16:56.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.984+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~XFRHIWUTYKOJ.2 size=440508 2024-05-09T20:16:56.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.988+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=XFRHIWUTYKOJ tag=8OZU64E2T13L type=write size=3586236 2024-05-09T20:16:56.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.992+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=G3YJAI9H5BEX type=del size=4050939 2024-05-09T20:16:56.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:56.996+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~LAFOJ0SW6ZHE.0 size=925748 2024-05-09T20:16:56.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.000+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=LAFOJ0SW6ZHE tag=QLYTIQOY0TQ6 type=write size=1974324 2024-05-09T20:16:57.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.008+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~1YD6KTW6DE2F.0 size=772938 2024-05-09T20:16:57.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.008+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=1YD6KTW6DE2F tag=Y7F22BAP3ARO type=write size=1821514 2024-05-09T20:16:57.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.012+0000 7fdc882e58c0 1 < completed write operation key=T5EL7X0P450L tag=Y58ZF83N1N6A type=write size=261309 2024-05-09T20:16:57.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.016+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=ACOVLBW94QIL type=write size=16231 2024-05-09T20:16:57.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.020+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=VFKNWUNNF7WK type=del 2024-05-09T20:16:57.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.024+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~TXMQD2VH6R5B.0 size=430580 2024-05-09T20:16:57.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.024+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=TXMQD2VH6R5B tag=UTIEYNSXPWHP type=write size=1479156 2024-05-09T20:16:57.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.028+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=MKQXMFR00NIB type=del 2024-05-09T20:16:57.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.032+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=F235KEBZMHNS type=del size=4007070 2024-05-09T20:16:57.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.036+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=8IXTE2GN0FO9 type=del 2024-05-09T20:16:57.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.040+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~TGBHMJ1LZ1KR.0 size=1048576 2024-05-09T20:16:57.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.044+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~TGBHMJ1LZ1KR.1 size=1048576 2024-05-09T20:16:57.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.044+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~TGBHMJ1LZ1KR.2 size=410746 2024-05-09T20:16:57.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.044+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=TGBHMJ1LZ1KR tag=3QOGPOJIGWX0 type=write size=3556474 2024-05-09T20:16:57.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.048+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=WQEKIHSPCGSK type=del 2024-05-09T20:16:57.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.052+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~PPIB201UO1WM.0 size=1048576 2024-05-09T20:16:57.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.056+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~PPIB201UO1WM.1 size=306334 2024-05-09T20:16:57.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.056+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=PPIB201UO1WM tag=36EWL97A2R2A type=write size=2403486 2024-05-09T20:16:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.060+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~39BQF4AG0TOS.0 size=1048576 2024-05-09T20:16:57.054 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.064+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~39BQF4AG0TOS.1 size=274358 2024-05-09T20:16:57.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.068+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=39BQF4AG0TOS tag=ZR6XJW8YS6U6 type=write size=2371510 2024-05-09T20:16:57.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.072+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=Y9SVHAF8HF8K type=del size=4129161 2024-05-09T20:16:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.076+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=ICWQCR2OZIGP type=del 2024-05-09T20:16:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.080+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~0W7H9ALL1UJB.0 size=1040935 2024-05-09T20:16:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.080+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=0W7H9ALL1UJB tag=FOASWVO2Y1N2 type=write size=2089511 2024-05-09T20:16:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.084+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~2A8CJX15C6Q4.0 size=372457 2024-05-09T20:16:57.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.088+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=2A8CJX15C6Q4 tag=ZH5J9XQLJ8CU type=write size=1421033 2024-05-09T20:16:57.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.092+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=KGL5QBJPZAKW type=del size=74553 2024-05-09T20:16:57.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.096+0000 7fdc882e58c0 1 < suggested update operation key=WC5YH0OU51WX tag=ZB77M1GIVL2E type=write size=798019 2024-05-09T20:16:57.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.104+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~J81LKFKQ6UUE.0 size=1048576 2024-05-09T20:16:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.104+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~J81LKFKQ6UUE.1 size=1048576 2024-05-09T20:16:57.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.108+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~J81LKFKQ6UUE.2 size=135454 2024-05-09T20:16:57.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.108+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=J81LKFKQ6UUE tag=XZIS6AKVYR2D type=write size=3281182 2024-05-09T20:16:57.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.112+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=YBHLWV460TFA type=del 2024-05-09T20:16:57.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.116+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~GTWIJIP0P3VK.0 size=1048576 2024-05-09T20:16:57.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.120+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~GTWIJIP0P3VK.1 size=290629 2024-05-09T20:16:57.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.120+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=GTWIJIP0P3VK tag=XV4EPQOB23TK type=write size=2387781 2024-05-09T20:16:57.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.124+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~K0JA3NCL5YJ0.0 size=1048576 2024-05-09T20:16:57.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.124+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~K0JA3NCL5YJ0.1 size=105174 2024-05-09T20:16:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.132+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=K0JA3NCL5YJ0 tag=1M01D59NRMFQ type=write size=2202326 2024-05-09T20:16:57.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.132+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=AXYMPCHORTKH type=del size=2363711 2024-05-09T20:16:57.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.140+0000 7fdc882e58c0 1 < suggested update operation key=T6T3IOW9VPA0 tag=ILCRF77MPSKP type=del size=2703024 2024-05-09T20:16:57.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.144+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=9MHOTRZ947NG type=del size=2849184 2024-05-09T20:16:57.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.148+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=YIQ9SUT7MIKI type=del 2024-05-09T20:16:57.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.152+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=DWK5OVOB9DVV type=del size=4114053 2024-05-09T20:16:57.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.156+0000 7fdc882e58c0 1 < suggested remove operation key=O07UHAIJZF0R tag=16Q7OO8305LZ type=write 2024-05-09T20:16:57.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.164+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~GQPE8TLO2GJ9.0 size=1048576 2024-05-09T20:16:57.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.168+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~GQPE8TLO2GJ9.1 size=576003 2024-05-09T20:16:57.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.168+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=GQPE8TLO2GJ9 tag=53OORNOK2V0Z type=write size=2673155 2024-05-09T20:16:57.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.168+0000 7fdc882e58c0 -1 < failed to complete operation key=O07UHAIJZF0R tag=G3YJAI9H5BEX type=del size=4050939: (22) Invalid argument 2024-05-09T20:16:57.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.172+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=2Q42BNFQSZNJ type=del size=446716 2024-05-09T20:16:57.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.176+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~1YD6KTW6DE2F.0 size=772938 2024-05-09T20:16:57.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.184+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=1YD6KTW6DE2F tag=Y7F22BAP3ARO type=write size=1821514 2024-05-09T20:16:57.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.188+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~KCXPQY31R0UT.0 size=1048576 2024-05-09T20:16:57.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.192+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~KCXPQY31R0UT.1 size=1048576 2024-05-09T20:16:57.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.192+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~KCXPQY31R0UT.2 size=58126 2024-05-09T20:16:57.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.192+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=KCXPQY31R0UT tag=Y9GF3608W6LZ type=write size=3203854 2024-05-09T20:16:57.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.196+0000 7fdc882e58c0 1 < canceled operation key=EK78CNKUHNKO tag=ACOVLBW94QIL type=write size=16231 2024-05-09T20:16:57.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.204+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~03FEJ0AYQY7B.0 size=1048576 2024-05-09T20:16:57.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.208+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~03FEJ0AYQY7B.1 size=1048576 2024-05-09T20:16:57.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.208+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~03FEJ0AYQY7B.2 size=190650 2024-05-09T20:16:57.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.208+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=03FEJ0AYQY7B tag=ALHPXTAYEQWE type=write size=3336378 2024-05-09T20:16:57.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.208+0000 7fdc882e58c0 1 < canceled part key=_multipart_5H2GWM9LQELZ.2~TXMQD2VH6R5B.0 size=430580 2024-05-09T20:16:57.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.220+0000 7fdc882e58c0 1 < canceled multipart upload key=5H2GWM9LQELZ upload=TXMQD2VH6R5B tag=UTIEYNSXPWHP type=write size=1479156 2024-05-09T20:16:57.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.224+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~F82EJ7A86UZU.0 size=1048576 2024-05-09T20:16:57.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.228+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~F82EJ7A86UZU.1 size=1048576 2024-05-09T20:16:57.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.232+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~F82EJ7A86UZU.2 size=68511 2024-05-09T20:16:57.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.232+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=F82EJ7A86UZU tag=PLZTHTH0QJ2Z type=write size=3214239 2024-05-09T20:16:57.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.236+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=F235KEBZMHNS type=del 2024-05-09T20:16:57.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.240+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=YX9PB10Y4N94 type=del size=1163724 2024-05-09T20:16:57.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.240+0000 7fdc882e58c0 1 < canceled part key=_multipart_AJANS5ADJO5B.2~TGBHMJ1LZ1KR.0 size=1048576 2024-05-09T20:16:57.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.244+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~TGBHMJ1LZ1KR.1 size=1048576 2024-05-09T20:16:57.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.248+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~TGBHMJ1LZ1KR.2 size=410746 2024-05-09T20:16:57.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.252+0000 7fdc882e58c0 1 < canceled multipart upload key=AJANS5ADJO5B upload=TGBHMJ1LZ1KR tag=3QOGPOJIGWX0 type=write size=3556474 2024-05-09T20:16:57.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.256+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=OXOFSBR4D5DB type=del size=3971598 2024-05-09T20:16:57.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.260+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~PPIB201UO1WM.0 size=1048576 2024-05-09T20:16:57.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.264+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~PPIB201UO1WM.1 size=306334 2024-05-09T20:16:57.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.268+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=PPIB201UO1WM tag=36EWL97A2R2A type=write size=2403486 2024-05-09T20:16:57.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.272+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=X2I8KK8BTNL7 type=del size=1680425 2024-05-09T20:16:57.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.276+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=Y9SVHAF8HF8K type=del 2024-05-09T20:16:57.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.280+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=SDHEM5U409OY type=del size=2788521 2024-05-09T20:16:57.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.280+0000 7fdc882e58c0 1 < canceled part key=_multipart_CNV7NHP27JZ7.2~0W7H9ALL1UJB.0 size=1040935 2024-05-09T20:16:57.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.288+0000 7fdc882e58c0 1 < canceled multipart upload key=CNV7NHP27JZ7 upload=0W7H9ALL1UJB tag=FOASWVO2Y1N2 type=write size=2089511 2024-05-09T20:16:57.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.292+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=JG6HPRFAIJFJ type=del size=2684416 2024-05-09T20:16:57.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.296+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=KGL5QBJPZAKW type=del 2024-05-09T20:16:57.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.304+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~3CAJ0YSCBM90.0 size=517629 2024-05-09T20:16:57.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.304+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=3CAJ0YSCBM90 tag=RAA18BBA4P25 type=write size=1566205 2024-05-09T20:16:57.300 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.308+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~J81LKFKQ6UUE.0 size=1048576 2024-05-09T20:16:57.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.312+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~J81LKFKQ6UUE.1 size=1048576 2024-05-09T20:16:57.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.312+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~J81LKFKQ6UUE.2 size=135454 2024-05-09T20:16:57.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.320+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=J81LKFKQ6UUE tag=XZIS6AKVYR2D type=write size=3281182 2024-05-09T20:16:57.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.324+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=9DYGAGO2JOL2 type=del size=1543276 2024-05-09T20:16:57.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.328+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~GTWIJIP0P3VK.0 size=1048576 2024-05-09T20:16:57.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.328+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~GTWIJIP0P3VK.1 size=290629 2024-05-09T20:16:57.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.336+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=GTWIJIP0P3VK tag=XV4EPQOB23TK type=write size=2387781 2024-05-09T20:16:57.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.340+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=M3C6OCEGMFKZ type=del size=3838201 2024-05-09T20:16:57.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.344+0000 7fdc882e58c0 1 < canceled operation key=AJANS5ADJO5B tag=AXYMPCHORTKH type=del size=2363711 2024-05-09T20:16:57.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.348+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~NMN908VZ117P.0 size=188876 2024-05-09T20:16:57.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.348+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=NMN908VZ117P tag=KEVX85YK7EJY type=write size=1237452 2024-05-09T20:16:57.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.352+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=9MHOTRZ947NG type=del 2024-05-09T20:16:57.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.356+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=VOT374K6B56C type=del size=4072579 2024-05-09T20:16:57.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.360+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=DWK5OVOB9DVV type=del 2024-05-09T20:16:57.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.364+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=VXQU3RJAWIXU type=write size=243871 2024-05-09T20:16:57.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.368+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~GQPE8TLO2GJ9.0 size=1048576 2024-05-09T20:16:57.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.372+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~GQPE8TLO2GJ9.1 size=576003 2024-05-09T20:16:57.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.376+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=GQPE8TLO2GJ9 tag=53OORNOK2V0Z type=write size=2673155 2024-05-09T20:16:57.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.380+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=TMT9HUGX3E0V type=del size=854863 2024-05-09T20:16:57.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.384+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=2Q42BNFQSZNJ type=del 2024-05-09T20:16:57.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.392+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~UWHA2TE1TZVD.0 size=1048576 2024-05-09T20:16:57.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.396+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~UWHA2TE1TZVD.1 size=435564 2024-05-09T20:16:57.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.396+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=UWHA2TE1TZVD tag=LL1UR8J0TIHO type=write size=2532716 2024-05-09T20:16:57.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.396+0000 7fdc882e58c0 1 < canceled part key=_multipart_T6T3IOW9VPA0.2~KCXPQY31R0UT.0 size=1048576 2024-05-09T20:16:57.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.396+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~KCXPQY31R0UT.1 size=1048576 2024-05-09T20:16:57.392 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.400+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~KCXPQY31R0UT.2 size=58126 2024-05-09T20:16:57.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.408+0000 7fdc882e58c0 1 < canceled multipart upload key=T6T3IOW9VPA0 upload=KCXPQY31R0UT tag=Y9GF3608W6LZ type=write size=3203854 2024-05-09T20:16:57.406 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.416+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~UA4DIN8KB3Z1.0 size=421464 2024-05-09T20:16:57.406 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.416+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=UA4DIN8KB3Z1 tag=ZIKJF41YQAYE type=write size=1470040 2024-05-09T20:16:57.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.416+0000 7fdc882e58c0 1 < canceled part key=_multipart_5UMJFJRKUPKR.2~03FEJ0AYQY7B.0 size=1048576 2024-05-09T20:16:57.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.416+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~03FEJ0AYQY7B.1 size=1048576 2024-05-09T20:16:57.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.420+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~03FEJ0AYQY7B.2 size=190650 2024-05-09T20:16:57.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.428+0000 7fdc882e58c0 1 < canceled multipart upload key=5UMJFJRKUPKR upload=03FEJ0AYQY7B tag=ALHPXTAYEQWE type=write size=3336378 2024-05-09T20:16:57.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.428+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=6Y9HHTQRJQED type=del size=1212734 2024-05-09T20:16:57.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.432+0000 7fdc882e58c0 1 < canceled part key=_multipart_IZPPFLWVPRKO.2~F82EJ7A86UZU.0 size=1048576 2024-05-09T20:16:57.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.432+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~F82EJ7A86UZU.1 size=1048576 2024-05-09T20:16:57.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.436+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~F82EJ7A86UZU.2 size=68511 2024-05-09T20:16:57.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.444+0000 7fdc882e58c0 1 < canceled multipart upload key=IZPPFLWVPRKO upload=F82EJ7A86UZU tag=PLZTHTH0QJ2Z type=write size=3214239 2024-05-09T20:16:57.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.448+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=JH6X2Q9IOCF0 type=del size=896873 2024-05-09T20:16:57.443 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.452+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=YX9PB10Y4N94 type=del 2024-05-09T20:16:57.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.456+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=HCRA3TUGS5CU type=del size=890209 2024-05-09T20:16:57.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.464+0000 7fdc882e58c0 1 < suggested update operation key=SNPNT5JFJOUS tag=OXOFSBR4D5DB type=del size=3971598 2024-05-09T20:16:57.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.468+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=VL2MHH2FJREO type=del size=3598594 2024-05-09T20:16:57.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.472+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=X2I8KK8BTNL7 type=del 2024-05-09T20:16:57.465 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.472+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=7507NH6VN132 type=write size=861142 2024-05-09T20:16:57.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.476+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=SDHEM5U409OY type=del 2024-05-09T20:16:57.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~2S0WJQWD28WE.0 size=1048576 2024-05-09T20:16:57.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~2S0WJQWD28WE.1 size=1048576 2024-05-09T20:16:57.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.488+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~2S0WJQWD28WE.2 size=312046 2024-05-09T20:16:57.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.488+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=2S0WJQWD28WE tag=0P2MSPUWQZ48 type=write size=3457774 2024-05-09T20:16:57.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.492+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=JG6HPRFAIJFJ type=del 2024-05-09T20:16:57.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.496+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=13XLBFNWF6M7 type=del size=2529325 2024-05-09T20:16:57.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.500+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~3CAJ0YSCBM90.0 size=517629 2024-05-09T20:16:57.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.504+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=3CAJ0YSCBM90 tag=RAA18BBA4P25 type=write size=1566205 2024-05-09T20:16:57.499 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.508+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=2NF933HD5LU0 type=del size=28204 2024-05-09T20:16:57.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.512+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=9DYGAGO2JOL2 type=del 2024-05-09T20:16:57.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.512+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=Z6V2ODIW5ILV type=write size=921837 2024-05-09T20:16:57.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.516+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=M3C6OCEGMFKZ type=del 2024-05-09T20:16:57.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.524+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~XLPB20FB9Q5N.0 size=856736 2024-05-09T20:16:57.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.524+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=XLPB20FB9Q5N tag=BRTP2SVP6GZW type=write size=1905312 2024-05-09T20:16:57.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.528+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~NMN908VZ117P.0 size=188876 2024-05-09T20:16:57.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.532+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=NMN908VZ117P tag=KEVX85YK7EJY type=write size=1237452 2024-05-09T20:16:57.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.536+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=323DMEQMNX7L type=del size=3170673 2024-05-09T20:16:57.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.540+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=VOT374K6B56C type=del 2024-05-09T20:16:57.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.544+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=1XB4BS7N7CLG type=write size=286219 2024-05-09T20:16:57.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.548+0000 7fdc882e58c0 1 < completed write operation key=TQ5ZHATLNVPP tag=VXQU3RJAWIXU type=write size=243871 2024-05-09T20:16:57.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.552+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~4SQSVVPY2WQ6.0 size=1048576 2024-05-09T20:16:57.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.560+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~4SQSVVPY2WQ6.1 size=246239 2024-05-09T20:16:57.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.560+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=4SQSVVPY2WQ6 tag=OFKJLFI0M4JB type=write size=2343391 2024-05-09T20:16:57.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.564+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=TMT9HUGX3E0V type=del 2024-05-09T20:16:57.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.568+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=PCN3L6GKOGNK type=write size=387050 2024-05-09T20:16:57.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.572+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~UWHA2TE1TZVD.0 size=1048576 2024-05-09T20:16:57.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.576+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~UWHA2TE1TZVD.1 size=435564 2024-05-09T20:16:57.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.580+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=UWHA2TE1TZVD tag=LL1UR8J0TIHO type=write size=2532716 2024-05-09T20:16:57.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.584+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=9QD9GQDBDF3E type=write size=147378 2024-05-09T20:16:57.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.588+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~UA4DIN8KB3Z1.0 size=421464 2024-05-09T20:16:57.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.592+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=UA4DIN8KB3Z1 tag=ZIKJF41YQAYE type=write size=1470040 2024-05-09T20:16:57.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.600+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~OJSNM373LU0W.0 size=965988 2024-05-09T20:16:57.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.600+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=OJSNM373LU0W tag=7O1HEEN8EPOU type=write size=2014564 2024-05-09T20:16:57.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.604+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=6Y9HHTQRJQED type=del 2024-05-09T20:16:57.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~TCO43ZU0QPG0.0 size=1048576 2024-05-09T20:16:57.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~TCO43ZU0QPG0.1 size=915421 2024-05-09T20:16:57.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.612+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=TCO43ZU0QPG0 tag=YLJC523SMD35 type=write size=3012573 2024-05-09T20:16:57.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.620+0000 7fdc882e58c0 1 < suggested update operation key=NNGWUSCETMH4 tag=JH6X2Q9IOCF0 type=del size=896873 2024-05-09T20:16:57.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.624+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=MTVKDPU6HZHH type=del size=2224234 2024-05-09T20:16:57.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.628+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=HCRA3TUGS5CU type=del 2024-05-09T20:16:57.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.632+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~4I91VTQHJ2GI.0 size=996038 2024-05-09T20:16:57.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.632+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=4I91VTQHJ2GI tag=U3X9K7CO4MU8 type=write size=2044614 2024-05-09T20:16:57.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.636+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=VL2MHH2FJREO type=del 2024-05-09T20:16:57.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.640+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=N31UEU5RGP2R type=del size=3625727 2024-05-09T20:16:57.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.648+0000 7fdc882e58c0 1 < suggested remove operation key=TQ5ZHATLNVPP tag=7507NH6VN132 type=write 2024-05-09T20:16:57.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.652+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=YYLZHRGO8DNJ type=del size=658308 2024-05-09T20:16:57.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.652+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~2S0WJQWD28WE.0 size=1048576 2024-05-09T20:16:57.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.656+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~2S0WJQWD28WE.1 size=1048576 2024-05-09T20:16:57.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.660+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~2S0WJQWD28WE.2 size=312046 2024-05-09T20:16:57.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.664+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=2S0WJQWD28WE tag=0P2MSPUWQZ48 type=write size=3457774 2024-05-09T20:16:57.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.672+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~17PXVTDL0MHF.0 size=1048576 2024-05-09T20:16:57.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.672+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~17PXVTDL0MHF.1 size=1048576 2024-05-09T20:16:57.668 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.676+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~17PXVTDL0MHF.2 size=485334 2024-05-09T20:16:57.668 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.676+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=17PXVTDL0MHF tag=KC7PQSJ75J1R type=write size=3631062 2024-05-09T20:16:57.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.680+0000 7fdc882e58c0 -1 < failed to complete operation key=NNGWUSCETMH4 tag=13XLBFNWF6M7 type=del size=2529325: (22) Invalid argument 2024-05-09T20:16:57.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.688+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~A88FOZ6D8KSI.0 size=732754 2024-05-09T20:16:57.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.688+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=A88FOZ6D8KSI tag=IQ0F0HMRZEKZ type=write size=1781330 2024-05-09T20:16:57.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.688+0000 7fdc882e58c0 1 < canceled operation key=T6T3IOW9VPA0 tag=2NF933HD5LU0 type=del size=28204 2024-05-09T20:16:57.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.696+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~42KM5D9QMJS7.0 size=1048576 2024-05-09T20:16:57.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.700+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~42KM5D9QMJS7.1 size=1048576 2024-05-09T20:16:57.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.700+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~42KM5D9QMJS7.2 size=268089 2024-05-09T20:16:57.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.700+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=42KM5D9QMJS7 tag=JX37U8179XGL type=write size=3413817 2024-05-09T20:16:57.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.704+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=Z6V2ODIW5ILV type=write size=921837 2024-05-09T20:16:57.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~7W0TUE00280Z.0 size=1048576 2024-05-09T20:16:57.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.716+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~7W0TUE00280Z.1 size=22859 2024-05-09T20:16:57.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.716+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=7W0TUE00280Z tag=NNW5R4DFJ4HP type=write size=2120011 2024-05-09T20:16:57.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.720+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~XLPB20FB9Q5N.0 size=856736 2024-05-09T20:16:57.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.724+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=XLPB20FB9Q5N tag=BRTP2SVP6GZW type=write size=1905312 2024-05-09T20:16:57.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.728+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=5RA8QC3TFI2J type=del size=1302625 2024-05-09T20:16:57.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.732+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=323DMEQMNX7L type=del 2024-05-09T20:16:57.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~IXN6OHDQK0C3.0 size=1048576 2024-05-09T20:16:57.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.740+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~IXN6OHDQK0C3.1 size=738179 2024-05-09T20:16:57.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.740+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=IXN6OHDQK0C3 tag=W66RCJUH2LHE type=write size=2835331 2024-05-09T20:16:57.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.748+0000 7fdc882e58c0 1 < suggested remove operation key=NYHR6GTPQYZS tag=1XB4BS7N7CLG type=write 2024-05-09T20:16:57.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.748+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=N8680C1L1467 type=del size=176527 2024-05-09T20:16:57.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.752+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~4SQSVVPY2WQ6.0 size=1048576 2024-05-09T20:16:57.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.756+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~4SQSVVPY2WQ6.1 size=246239 2024-05-09T20:16:57.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.760+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=4SQSVVPY2WQ6 tag=OFKJLFI0M4JB type=write size=2343391 2024-05-09T20:16:57.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.764+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~MXH6DQJPK1AL.0 size=156409 2024-05-09T20:16:57.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.764+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=MXH6DQJPK1AL tag=CX373K456OQ5 type=write size=1204985 2024-05-09T20:16:57.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.768+0000 7fdc882e58c0 1 < canceled operation key=346HV1XQKCOZ tag=PCN3L6GKOGNK type=write size=387050 2024-05-09T20:16:57.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.772+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=NJ7HPHDWK52F type=del size=535709 2024-05-09T20:16:57.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.776+0000 7fdc882e58c0 1 < completed write operation key=G0DX1F257RLD tag=9QD9GQDBDF3E type=write size=147378 2024-05-09T20:16:57.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~0Z1GGJSN45PU.0 size=1048576 2024-05-09T20:16:57.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~0Z1GGJSN45PU.1 size=1048576 2024-05-09T20:16:57.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~0Z1GGJSN45PU.2 size=939557 2024-05-09T20:16:57.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.784+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=0Z1GGJSN45PU tag=ZP14A6SWZ2VE type=write size=4085285 2024-05-09T20:16:57.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.788+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~OJSNM373LU0W.0 size=965988 2024-05-09T20:16:57.786 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.792+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=OJSNM373LU0W tag=7O1HEEN8EPOU type=write size=2014564 2024-05-09T20:16:57.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.800+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~2OZT2ZJM8I80.0 size=1048576 2024-05-09T20:16:57.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.804+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~2OZT2ZJM8I80.1 size=543630 2024-05-09T20:16:57.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.804+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=2OZT2ZJM8I80 tag=3C22YDQEDAN0 type=write size=2640782 2024-05-09T20:16:57.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.804+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~TCO43ZU0QPG0.0 size=1048576 2024-05-09T20:16:57.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.808+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~TCO43ZU0QPG0.1 size=915421 2024-05-09T20:16:57.806 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.812+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=TCO43ZU0QPG0 tag=YLJC523SMD35 type=write size=3012573 2024-05-09T20:16:57.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.820+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~9TDZ3QNSWV48.0 size=1048576 2024-05-09T20:16:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.820+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~9TDZ3QNSWV48.1 size=606682 2024-05-09T20:16:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.820+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=9TDZ3QNSWV48 tag=DSNCY46Y6GQY type=write size=2703834 2024-05-09T20:16:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.824+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=MTVKDPU6HZHH type=del 2024-05-09T20:16:57.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.828+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=F1NXECFEY27E type=del size=58458 2024-05-09T20:16:57.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.832+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~4I91VTQHJ2GI.0 size=996038 2024-05-09T20:16:57.828 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.836+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=4I91VTQHJ2GI tag=U3X9K7CO4MU8 type=write size=2044614 2024-05-09T20:16:57.830 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.840+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=717PE1YX31WM type=del size=742407 2024-05-09T20:16:57.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.840+0000 7fdc882e58c0 1 < canceled operation key=BS0T1OQOBS67 tag=N31UEU5RGP2R type=del size=3625727 2024-05-09T20:16:57.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.848+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~C6EIMD29R9DA.0 size=1048576 2024-05-09T20:16:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.848+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~C6EIMD29R9DA.1 size=520574 2024-05-09T20:16:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.848+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=C6EIMD29R9DA tag=OVE9U4C32MMU type=write size=2617726 2024-05-09T20:16:57.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.852+0000 7fdc882e58c0 1 < canceled operation key=T6T3IOW9VPA0 tag=YYLZHRGO8DNJ type=del size=658308 2024-05-09T20:16:57.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.856+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=E3BEBSX5G398 type=del size=4151458 2024-05-09T20:16:57.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.856+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~17PXVTDL0MHF.0 size=1048576 2024-05-09T20:16:57.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.860+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~17PXVTDL0MHF.1 size=1048576 2024-05-09T20:16:57.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.864+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~17PXVTDL0MHF.2 size=485334 2024-05-09T20:16:57.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.868+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=17PXVTDL0MHF tag=KC7PQSJ75J1R type=write size=3631062 2024-05-09T20:16:57.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.872+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=2OWWD1R0AEH9 type=del size=2823787 2024-05-09T20:16:57.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.872+0000 7fdc882e58c0 1 < canceled part key=_multipart_8FFZZDURUWM1.2~A88FOZ6D8KSI.0 size=732754 2024-05-09T20:16:57.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.876+0000 7fdc882e58c0 1 < canceled multipart upload key=8FFZZDURUWM1 upload=A88FOZ6D8KSI tag=IQ0F0HMRZEKZ type=write size=1781330 2024-05-09T20:16:57.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.884+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~5KDK2ND84E2G.0 size=1048576 2024-05-09T20:16:57.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.884+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~5KDK2ND84E2G.1 size=856249 2024-05-09T20:16:57.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.884+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=5KDK2ND84E2G tag=SAVJ7OAEBAJK type=write size=2953401 2024-05-09T20:16:57.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.888+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~42KM5D9QMJS7.0 size=1048576 2024-05-09T20:16:57.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.892+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~42KM5D9QMJS7.1 size=1048576 2024-05-09T20:16:57.884 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.892+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~42KM5D9QMJS7.2 size=268089 2024-05-09T20:16:57.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.896+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=42KM5D9QMJS7 tag=JX37U8179XGL type=write size=3413817 2024-05-09T20:16:57.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.900+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=IK7B3Q2KXVDW type=del size=3418933 2024-05-09T20:16:57.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.904+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~7W0TUE00280Z.0 size=1048576 2024-05-09T20:16:57.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.904+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~7W0TUE00280Z.1 size=22859 2024-05-09T20:16:57.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.908+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=7W0TUE00280Z tag=NNW5R4DFJ4HP type=write size=2120011 2024-05-09T20:16:57.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.912+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=EZCGLTVEKNJO type=del size=809293 2024-05-09T20:16:57.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.916+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=5RA8QC3TFI2J type=del 2024-05-09T20:16:57.911 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.920+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~VCES57KZ6AJ9.0 size=559299 2024-05-09T20:16:57.911 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.920+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=VCES57KZ6AJ9 tag=731I3V8M9I4A type=write size=1607875 2024-05-09T20:16:57.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.924+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~IXN6OHDQK0C3.0 size=1048576 2024-05-09T20:16:57.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.924+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~IXN6OHDQK0C3.1 size=738179 2024-05-09T20:16:57.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.928+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=IXN6OHDQK0C3 tag=W66RCJUH2LHE type=write size=2835331 2024-05-09T20:16:57.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~HR1RNMUVSRKE.0 size=585016 2024-05-09T20:16:57.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.936+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=HR1RNMUVSRKE tag=TPWT6O427L1P type=write size=1633592 2024-05-09T20:16:57.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.940+0000 7fdc882e58c0 1 < suggested update operation key=P2DNA4DGI54P tag=N8680C1L1467 type=del size=176527 2024-05-09T20:16:57.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.944+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=B2D4U3F8BI6U type=del size=2247271 2024-05-09T20:16:57.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.944+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~MXH6DQJPK1AL.0 size=156409 2024-05-09T20:16:57.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.948+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=MXH6DQJPK1AL tag=CX373K456OQ5 type=write size=1204985 2024-05-09T20:16:57.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.952+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=2Z23KEOWTS0I type=del size=2203796 2024-05-09T20:16:57.946 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.956+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=NJ7HPHDWK52F type=del 2024-05-09T20:16:57.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.960+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~B3PIV2PFD1H5.0 size=1048576 2024-05-09T20:16:57.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.960+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~B3PIV2PFD1H5.1 size=488056 2024-05-09T20:16:57.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.960+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=B3PIV2PFD1H5 tag=JHDFRQXV71NY type=write size=2585208 2024-05-09T20:16:57.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.964+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~0Z1GGJSN45PU.0 size=1048576 2024-05-09T20:16:57.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.968+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~0Z1GGJSN45PU.1 size=1048576 2024-05-09T20:16:57.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.968+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~0Z1GGJSN45PU.2 size=939557 2024-05-09T20:16:57.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.972+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=0Z1GGJSN45PU tag=ZP14A6SWZ2VE type=write size=4085285 2024-05-09T20:16:57.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.976+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=XF96YGB4M0ED type=del size=1228959 2024-05-09T20:16:57.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.980+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~2OZT2ZJM8I80.0 size=1048576 2024-05-09T20:16:57.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.980+0000 7fdc882e58c0 1 < canceled part key=_multipart_SPXHCWZICCWY.2~2OZT2ZJM8I80.1 size=543630 2024-05-09T20:16:57.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.984+0000 7fdc882e58c0 1 < canceled multipart upload key=SPXHCWZICCWY upload=2OZT2ZJM8I80 tag=3C22YDQEDAN0 type=write size=2640782 2024-05-09T20:16:57.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.984+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=QACMQY7QF7WO type=write size=498456 2024-05-09T20:16:57.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.988+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~9TDZ3QNSWV48.0 size=1048576 2024-05-09T20:16:57.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.992+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~9TDZ3QNSWV48.1 size=606682 2024-05-09T20:16:57.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.996+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=9TDZ3QNSWV48 tag=DSNCY46Y6GQY type=write size=2703834 2024-05-09T20:16:57.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:57.996+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=9Y0SUOLQG1IL type=write size=582302 2024-05-09T20:16:57.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.000+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=F1NXECFEY27E type=del 2024-05-09T20:16:57.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.004+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=S8EX7SCDR9VO type=del size=3134452 2024-05-09T20:16:57.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.004+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=717PE1YX31WM type=del 2024-05-09T20:16:58.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.008+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=6NF5847FJTK7 type=write size=863650 2024-05-09T20:16:58.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.012+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~C6EIMD29R9DA.0 size=1048576 2024-05-09T20:16:58.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.016+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~C6EIMD29R9DA.1 size=520574 2024-05-09T20:16:58.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.020+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=C6EIMD29R9DA tag=OVE9U4C32MMU type=write size=2617726 2024-05-09T20:16:58.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.020+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=NIJ63PSZUMHN type=del size=3554614 2024-05-09T20:16:58.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.024+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=E3BEBSX5G398 type=del 2024-05-09T20:16:58.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.028+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~D30PXMMYIGHX.0 size=714423 2024-05-09T20:16:58.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.028+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=D30PXMMYIGHX tag=RWO5S2BU9LC3 type=write size=1762999 2024-05-09T20:16:58.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.032+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=2OWWD1R0AEH9 type=del 2024-05-09T20:16:58.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.036+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~2TD1G05ZVW0Z.0 size=1048576 2024-05-09T20:16:58.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.040+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~2TD1G05ZVW0Z.1 size=1048576 2024-05-09T20:16:58.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.044+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~2TD1G05ZVW0Z.2 size=433669 2024-05-09T20:16:58.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.044+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=2TD1G05ZVW0Z tag=99F6XZKJNX8F type=write size=3579397 2024-05-09T20:16:58.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.044+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~5KDK2ND84E2G.0 size=1048576 2024-05-09T20:16:58.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.048+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~5KDK2ND84E2G.1 size=856249 2024-05-09T20:16:58.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.052+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=5KDK2ND84E2G tag=SAVJ7OAEBAJK type=write size=2953401 2024-05-09T20:16:58.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.056+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=04BWHYM6R5BC type=del size=415706 2024-05-09T20:16:58.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.060+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=IK7B3Q2KXVDW type=del 2024-05-09T20:16:58.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.064+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~NEW2KJPKOATT.0 size=899080 2024-05-09T20:16:58.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.064+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=NEW2KJPKOATT tag=2O60K9ZZ9W3P type=write size=1947656 2024-05-09T20:16:58.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.072+0000 7fdc882e58c0 1 < suggested remove operation key=JS90W6PIREMD tag=EZCGLTVEKNJO type=del 2024-05-09T20:16:58.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.072+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=HIW1G63HZH8V type=del size=2145700 2024-05-09T20:16:58.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.076+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~VCES57KZ6AJ9.0 size=559299 2024-05-09T20:16:58.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.080+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=VCES57KZ6AJ9 tag=731I3V8M9I4A type=write size=1607875 2024-05-09T20:16:58.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.084+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=81SK9XN4YEXJ type=del size=1697736 2024-05-09T20:16:58.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.088+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~HR1RNMUVSRKE.0 size=585016 2024-05-09T20:16:58.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.096+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=HR1RNMUVSRKE tag=TPWT6O427L1P type=write size=1633592 2024-05-09T20:16:58.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.100+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=B4HUMJO1FW3I type=del size=3947051 2024-05-09T20:16:58.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.100+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=B2D4U3F8BI6U type=del 2024-05-09T20:16:58.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.104+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=ECTSRM6J6WEP type=del size=2186169 2024-05-09T20:16:58.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.108+0000 7fdc882e58c0 1 < canceled operation key=54R9XPO62J0M tag=2Z23KEOWTS0I type=del size=2203796 2024-05-09T20:16:58.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.116+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~3DTSP6X3MJU9.0 size=1048576 2024-05-09T20:16:58.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.116+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~3DTSP6X3MJU9.1 size=1048576 2024-05-09T20:16:58.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.120+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~3DTSP6X3MJU9.2 size=663432 2024-05-09T20:16:58.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.120+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=3DTSP6X3MJU9 tag=D1GR1AIR8GKA type=write size=3809160 2024-05-09T20:16:58.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.124+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~B3PIV2PFD1H5.0 size=1048576 2024-05-09T20:16:58.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.124+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~B3PIV2PFD1H5.1 size=488056 2024-05-09T20:16:58.127 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.136+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=B3PIV2PFD1H5 tag=JHDFRQXV71NY type=write size=2585208 2024-05-09T20:16:58.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.136+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=WF4MLVHVT7P9 type=del size=2734867 2024-05-09T20:16:58.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.140+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=XF96YGB4M0ED type=del 2024-05-09T20:16:58.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.144+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=8VHBUFTHWL13 type=del size=3129441 2024-05-09T20:16:58.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.148+0000 7fdc882e58c0 1 < completed write operation key=O07UHAIJZF0R tag=QACMQY7QF7WO type=write size=498456 2024-05-09T20:16:58.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.148+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=MM61THWP2LMA type=del size=948603 2024-05-09T20:16:58.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.152+0000 7fdc882e58c0 1 < completed write operation key=T5EL7X0P450L tag=9Y0SUOLQG1IL type=write size=582302 2024-05-09T20:16:58.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.156+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=8CNS5X2GPKZ7 type=del size=2693089 2024-05-09T20:16:58.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.156+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=S8EX7SCDR9VO type=del 2024-05-09T20:16:58.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.164+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~WE0MI0JO2ZSD.0 size=1048576 2024-05-09T20:16:58.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.168+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~WE0MI0JO2ZSD.1 size=9138 2024-05-09T20:16:58.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.168+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=WE0MI0JO2ZSD tag=POD2UY69Q6B1 type=write size=2106290 2024-05-09T20:16:58.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.168+0000 7fdc882e58c0 1 < completed write operation key=AJANS5ADJO5B tag=6NF5847FJTK7 type=write size=863650 2024-05-09T20:16:58.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.172+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=UEDL1UULKIOB type=del size=190948 2024-05-09T20:16:58.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.176+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=NIJ63PSZUMHN type=del 2024-05-09T20:16:58.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.180+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=D3R7QF19A56F type=write size=482821 2024-05-09T20:16:58.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.184+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~D30PXMMYIGHX.0 size=714423 2024-05-09T20:16:58.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.188+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=D30PXMMYIGHX tag=RWO5S2BU9LC3 type=write size=1762999 2024-05-09T20:16:58.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.192+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=ET9HMRAMVCEB type=del size=1218691 2024-05-09T20:16:58.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.196+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~2TD1G05ZVW0Z.0 size=1048576 2024-05-09T20:16:58.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.196+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~2TD1G05ZVW0Z.1 size=1048576 2024-05-09T20:16:58.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.200+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~2TD1G05ZVW0Z.2 size=433669 2024-05-09T20:16:58.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.204+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=2TD1G05ZVW0Z tag=99F6XZKJNX8F type=write size=3579397 2024-05-09T20:16:58.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.208+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=IEZVXTCA8DXY type=del size=2038749 2024-05-09T20:16:58.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.212+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=04BWHYM6R5BC type=del 2024-05-09T20:16:58.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.216+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=QL8LB58CFRQS type=del size=993072 2024-05-09T20:16:58.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.220+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~NEW2KJPKOATT.0 size=899080 2024-05-09T20:16:58.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.224+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=NEW2KJPKOATT tag=2O60K9ZZ9W3P type=write size=1947656 2024-05-09T20:16:58.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.228+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=XUOHE3VKJA59 type=del size=1853884 2024-05-09T20:16:58.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.236+0000 7fdc882e58c0 1 < suggested update operation key=OGVA2778MJWU tag=HIW1G63HZH8V type=del size=2145700 2024-05-09T20:16:58.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.240+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=TRU37AF4NDW8 type=del size=2091195 2024-05-09T20:16:58.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.248+0000 7fdc882e58c0 1 < suggested remove operation key=WC5YH0OU51WX tag=81SK9XN4YEXJ type=del 2024-05-09T20:16:58.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.252+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=Z82C5XB4ZZ68 type=del size=3243721 2024-05-09T20:16:58.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.256+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=B4HUMJO1FW3I type=del 2024-05-09T20:16:58.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.260+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~5AO1LM8C1QIS.0 size=1048576 2024-05-09T20:16:58.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.264+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~5AO1LM8C1QIS.1 size=1048576 2024-05-09T20:16:58.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.268+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~5AO1LM8C1QIS.2 size=439889 2024-05-09T20:16:58.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.268+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=5AO1LM8C1QIS tag=JWA3OGVHEJ1A type=write size=3585617 2024-05-09T20:16:58.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.272+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=ECTSRM6J6WEP type=del 2024-05-09T20:16:58.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.272+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=21BY5MQMOW7B type=del size=1130590 2024-05-09T20:16:58.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.276+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~3DTSP6X3MJU9.0 size=1048576 2024-05-09T20:16:58.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.280+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~3DTSP6X3MJU9.1 size=1048576 2024-05-09T20:16:58.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.284+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~3DTSP6X3MJU9.2 size=663432 2024-05-09T20:16:58.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.288+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=3DTSP6X3MJU9 tag=D1GR1AIR8GKA type=write size=3809160 2024-05-09T20:16:58.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.296+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~X9GM69BR4XYQ.0 size=1048576 2024-05-09T20:16:58.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.300+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~X9GM69BR4XYQ.1 size=1048576 2024-05-09T20:16:58.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.300+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~X9GM69BR4XYQ.2 size=1005000 2024-05-09T20:16:58.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.300+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=X9GM69BR4XYQ tag=7TB4M0LZY0MS type=write size=4150728 2024-05-09T20:16:58.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.304+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=WF4MLVHVT7P9 type=del 2024-05-09T20:16:58.300 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.308+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=S2RKCI0TEZDD type=del size=528989 2024-05-09T20:16:58.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.312+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=8VHBUFTHWL13 type=del 2024-05-09T20:16:58.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.316+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~F5H1SRVTR4LN.0 size=1048576 2024-05-09T20:16:58.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.320+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~F5H1SRVTR4LN.1 size=1048576 2024-05-09T20:16:58.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.324+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~F5H1SRVTR4LN.2 size=766612 2024-05-09T20:16:58.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.324+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=F5H1SRVTR4LN tag=TI61ID8AQT3Q type=write size=3912340 2024-05-09T20:16:58.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.324+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=MM61THWP2LMA type=del 2024-05-09T20:16:58.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.328+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=S6GD48OUME4P type=del size=3861239 2024-05-09T20:16:58.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.336+0000 7fdc882e58c0 1 < suggested update operation key=OGVA2778MJWU tag=8CNS5X2GPKZ7 type=del size=2693089 2024-05-09T20:16:58.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.340+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=1F4LDGE83VMN type=write size=391841 2024-05-09T20:16:58.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.340+0000 7fdc882e58c0 1 < canceled part key=_multipart_TQ5ZHATLNVPP.2~WE0MI0JO2ZSD.0 size=1048576 2024-05-09T20:16:58.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.344+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~WE0MI0JO2ZSD.1 size=9138 2024-05-09T20:16:58.339 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.348+0000 7fdc882e58c0 1 < canceled multipart upload key=TQ5ZHATLNVPP upload=WE0MI0JO2ZSD tag=POD2UY69Q6B1 type=write size=2106290 2024-05-09T20:16:58.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.352+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=BOCO4CGN32YT type=del size=1904725 2024-05-09T20:16:58.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.352+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=UEDL1UULKIOB type=del 2024-05-09T20:16:58.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.356+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=MR3GGMHIHPTA type=del size=3763348 2024-05-09T20:16:58.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.360+0000 7fdc882e58c0 1 < completed write operation key=NYHR6GTPQYZS tag=D3R7QF19A56F type=write size=482821 2024-05-09T20:16:58.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.364+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=Q4WN4OXLLOSW type=del size=2198308 2024-05-09T20:16:58.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.368+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=ET9HMRAMVCEB type=del 2024-05-09T20:16:58.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.372+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=0D8CAI4HZJN4 type=del size=2102587 2024-05-09T20:16:58.367 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.376+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=IEZVXTCA8DXY type=del 2024-05-09T20:16:58.371 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.380+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=5ZY27LRCQOP1 type=del size=1432668 2024-05-09T20:16:58.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.384+0000 7fdc882e58c0 -1 < failed to complete operation key=OGVA2778MJWU tag=QL8LB58CFRQS type=del size=993072: (22) Invalid argument 2024-05-09T20:16:58.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.388+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~1J47UUY647OG.0 size=1048576 2024-05-09T20:16:58.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.392+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~1J47UUY647OG.1 size=622495 2024-05-09T20:16:58.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.392+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=1J47UUY647OG tag=R0P6P8X2ANL9 type=write size=2719647 2024-05-09T20:16:58.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.396+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=XUOHE3VKJA59 type=del 2024-05-09T20:16:58.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.396+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=SJC6GSLOA4D4 type=write size=529257 2024-05-09T20:16:58.392 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.400+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=TRU37AF4NDW8 type=del 2024-05-09T20:16:58.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.404+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=J332FUHD9VHZ type=write size=565156 2024-05-09T20:16:58.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.412+0000 7fdc882e58c0 1 < suggested remove operation key=NYHR6GTPQYZS tag=Z82C5XB4ZZ68 type=del 2024-05-09T20:16:58.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.416+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~JUKASSKG6KX1.0 size=1048576 2024-05-09T20:16:58.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.420+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~JUKASSKG6KX1.1 size=1048576 2024-05-09T20:16:58.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~JUKASSKG6KX1.2 size=163917 2024-05-09T20:16:58.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.424+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=JUKASSKG6KX1 tag=SW64VQTS3D81 type=write size=3309645 2024-05-09T20:16:58.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.424+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~5AO1LM8C1QIS.0 size=1048576 2024-05-09T20:16:58.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.424+0000 7fdc882e58c0 1 < canceled part key=_multipart_T5EL7X0P450L.2~5AO1LM8C1QIS.1 size=1048576 2024-05-09T20:16:58.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.428+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~5AO1LM8C1QIS.2 size=439889 2024-05-09T20:16:58.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.432+0000 7fdc882e58c0 1 < canceled multipart upload key=T5EL7X0P450L upload=5AO1LM8C1QIS tag=JWA3OGVHEJ1A type=write size=3585617 2024-05-09T20:16:58.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.436+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=4T97IQEV4C1L type=del size=454040 2024-05-09T20:16:58.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.440+0000 7fdc882e58c0 -1 < failed to complete operation key=OGVA2778MJWU tag=21BY5MQMOW7B type=del size=1130590: (22) Invalid argument 2024-05-09T20:16:58.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.448+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~DBNPXC5KI32S.0 size=732627 2024-05-09T20:16:58.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.448+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=DBNPXC5KI32S tag=K1ZPBD4U51FY type=write size=1781203 2024-05-09T20:16:58.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.452+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~X9GM69BR4XYQ.0 size=1048576 2024-05-09T20:16:58.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.456+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~X9GM69BR4XYQ.1 size=1048576 2024-05-09T20:16:58.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.456+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~X9GM69BR4XYQ.2 size=1005000 2024-05-09T20:16:58.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.464+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=X9GM69BR4XYQ tag=7TB4M0LZY0MS type=write size=4150728 2024-05-09T20:16:58.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.468+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~NZ8T7YHRSL79.0 size=45951 2024-05-09T20:16:58.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.468+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=NZ8T7YHRSL79 tag=3USZJ7VMVY82 type=write size=1094527 2024-05-09T20:16:58.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.472+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=S2RKCI0TEZDD type=del 2024-05-09T20:16:58.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.476+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=JIW6EBZAA24V type=write size=433114 2024-05-09T20:16:58.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.480+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~F5H1SRVTR4LN.0 size=1048576 2024-05-09T20:16:58.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.480+0000 7fdc882e58c0 1 < canceled part key=_multipart_VZ7XOAT899GU.2~F5H1SRVTR4LN.1 size=1048576 2024-05-09T20:16:58.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.480+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~F5H1SRVTR4LN.2 size=766612 2024-05-09T20:16:58.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.488+0000 7fdc882e58c0 1 < canceled multipart upload key=VZ7XOAT899GU upload=F5H1SRVTR4LN tag=TI61ID8AQT3Q type=write size=3912340 2024-05-09T20:16:58.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.492+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~JTOVAYXSNQFY.0 size=1048576 2024-05-09T20:16:58.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.496+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~JTOVAYXSNQFY.1 size=1048576 2024-05-09T20:16:58.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.500+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~JTOVAYXSNQFY.2 size=366256 2024-05-09T20:16:58.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.500+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=JTOVAYXSNQFY tag=DEZ5U04VWBY6 type=write size=3511984 2024-05-09T20:16:58.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.504+0000 7fdc882e58c0 -1 < failed to complete operation key=NYHR6GTPQYZS tag=S6GD48OUME4P type=del size=3861239: (22) Invalid argument 2024-05-09T20:16:58.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.508+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~KFY9NA0E08OV.0 size=1048576 2024-05-09T20:16:58.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.512+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~KFY9NA0E08OV.1 size=1048576 2024-05-09T20:16:58.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.512+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~KFY9NA0E08OV.2 size=259298 2024-05-09T20:16:58.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.512+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=KFY9NA0E08OV tag=ZR5V2GRZ27RG type=write size=3405026 2024-05-09T20:16:58.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.516+0000 7fdc882e58c0 1 < completed write operation key=TWWOL5D8C1MQ tag=1F4LDGE83VMN type=write size=391841 2024-05-09T20:16:58.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.520+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=YFYC5H2V7VIL type=del size=1920266 2024-05-09T20:16:58.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.524+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=BOCO4CGN32YT type=del 2024-05-09T20:16:58.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.528+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=T2AU8BMD9TRX type=del size=2430586 2024-05-09T20:16:58.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.532+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=MR3GGMHIHPTA type=del 2024-05-09T20:16:58.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.536+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~YLTUPUGAZ0QX.0 size=1048576 2024-05-09T20:16:58.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.540+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~YLTUPUGAZ0QX.1 size=1048576 2024-05-09T20:16:58.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.540+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~YLTUPUGAZ0QX.2 size=818274 2024-05-09T20:16:58.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.540+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=YLTUPUGAZ0QX tag=PHO2AE54XZZF type=write size=3964002 2024-05-09T20:16:58.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.548+0000 7fdc882e58c0 1 < suggested update operation key=8FFZZDURUWM1 tag=Q4WN4OXLLOSW type=del size=2198308 2024-05-09T20:16:58.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.552+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=U3TUSOWU3VUF type=write size=257614 2024-05-09T20:16:58.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.556+0000 7fdc882e58c0 1 < suggested remove operation key=WC5YH0OU51WX tag=0D8CAI4HZJN4 type=del 2024-05-09T20:16:58.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.564+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~KGOKGMF8F4YP.0 size=1048576 2024-05-09T20:16:58.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.564+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~KGOKGMF8F4YP.1 size=427291 2024-05-09T20:16:58.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.564+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=KGOKGMF8F4YP tag=S5K1SP12Y444 type=write size=2524443 2024-05-09T20:16:58.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.568+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=5ZY27LRCQOP1 type=del 2024-05-09T20:16:58.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.572+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=0NMCW43XUOAP type=del size=2256215 2024-05-09T20:16:58.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.576+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~1J47UUY647OG.0 size=1048576 2024-05-09T20:16:58.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.580+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~1J47UUY647OG.1 size=622495 2024-05-09T20:16:58.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.584+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=1J47UUY647OG tag=R0P6P8X2ANL9 type=write size=2719647 2024-05-09T20:16:58.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.592+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~1JWG3W2Z5QLN.0 size=1048576 2024-05-09T20:16:58.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.596+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~1JWG3W2Z5QLN.1 size=1048576 2024-05-09T20:16:58.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.596+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~1JWG3W2Z5QLN.2 size=406556 2024-05-09T20:16:58.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.596+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=1JWG3W2Z5QLN tag=IR220Y25O4RG type=write size=3552284 2024-05-09T20:16:58.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.600+0000 7fdc882e58c0 1 < completed write operation key=CNV7NHP27JZ7 tag=SJC6GSLOA4D4 type=write size=529257 2024-05-09T20:16:58.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.608+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~SAW83TQQFIJ5.0 size=911752 2024-05-09T20:16:58.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.608+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=SAW83TQQFIJ5 tag=OWCL8BNW9WAX type=write size=1960328 2024-05-09T20:16:58.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.612+0000 7fdc882e58c0 1 < completed write operation key=EK78CNKUHNKO tag=J332FUHD9VHZ type=write size=565156 2024-05-09T20:16:58.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.616+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~DONVT7QIU50S.0 size=1048576 2024-05-09T20:16:58.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.620+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~DONVT7QIU50S.1 size=887838 2024-05-09T20:16:58.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.620+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=DONVT7QIU50S tag=K31JQE4KF9UK type=write size=2984990 2024-05-09T20:16:58.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.624+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~JUKASSKG6KX1.0 size=1048576 2024-05-09T20:16:58.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.624+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~JUKASSKG6KX1.1 size=1048576 2024-05-09T20:16:58.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.628+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~JUKASSKG6KX1.2 size=163917 2024-05-09T20:16:58.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.636+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=JUKASSKG6KX1 tag=SW64VQTS3D81 type=write size=3309645 2024-05-09T20:16:58.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.640+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~YE67HL7X6XX7.0 size=1048576 2024-05-09T20:16:58.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~YE67HL7X6XX7.1 size=1048576 2024-05-09T20:16:58.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~YE67HL7X6XX7.2 size=834944 2024-05-09T20:16:58.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.644+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=YE67HL7X6XX7 tag=GHR99XDYJSY7 type=write size=3980672 2024-05-09T20:16:58.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.648+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=4T97IQEV4C1L type=del 2024-05-09T20:16:58.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.652+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=24DFQCZPCZBD type=del size=783118 2024-05-09T20:16:58.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.656+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~DBNPXC5KI32S.0 size=732627 2024-05-09T20:16:58.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.664+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=DBNPXC5KI32S tag=K1ZPBD4U51FY type=write size=1781203 2024-05-09T20:16:58.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.668+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=NE0V85BNLOXS type=del size=987872 2024-05-09T20:16:58.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.668+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~NZ8T7YHRSL79.0 size=45951 2024-05-09T20:16:58.667 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.676+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=NZ8T7YHRSL79 tag=3USZJ7VMVY82 type=write size=1094527 2024-05-09T20:16:58.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.680+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~DM0OLOBWCZQF.0 size=378597 2024-05-09T20:16:58.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.680+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=DM0OLOBWCZQF tag=BOTTWG86Z14W type=write size=1427173 2024-05-09T20:16:58.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.684+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=JIW6EBZAA24V type=write size=433114 2024-05-09T20:16:58.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.688+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=4T7I7IJVU2L8 type=del size=2000119 2024-05-09T20:16:58.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.688+0000 7fdc882e58c0 1 < canceled part key=_multipart_JS90W6PIREMD.2~JTOVAYXSNQFY.0 size=1048576 2024-05-09T20:16:58.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.692+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~JTOVAYXSNQFY.1 size=1048576 2024-05-09T20:16:58.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.692+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~JTOVAYXSNQFY.2 size=366256 2024-05-09T20:16:58.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.696+0000 7fdc882e58c0 1 < canceled multipart upload key=JS90W6PIREMD upload=JTOVAYXSNQFY tag=DEZ5U04VWBY6 type=write size=3511984 2024-05-09T20:16:58.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.700+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=1B7E8QHROUQK type=del size=461335 2024-05-09T20:16:58.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.700+0000 7fdc882e58c0 1 < canceled part key=_multipart_SNPNT5JFJOUS.2~KFY9NA0E08OV.0 size=1048576 2024-05-09T20:16:58.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.700+0000 7fdc882e58c0 1 < canceled part key=_multipart_SNPNT5JFJOUS.2~KFY9NA0E08OV.1 size=1048576 2024-05-09T20:16:58.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.704+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~KFY9NA0E08OV.2 size=259298 2024-05-09T20:16:58.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.708+0000 7fdc882e58c0 1 < canceled multipart upload key=SNPNT5JFJOUS upload=KFY9NA0E08OV tag=ZR5V2GRZ27RG type=write size=3405026 2024-05-09T20:16:58.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.716+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~T32FUUQX25GA.0 size=1048576 2024-05-09T20:16:58.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.720+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~T32FUUQX25GA.1 size=1048576 2024-05-09T20:16:58.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.724+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~T32FUUQX25GA.2 size=309976 2024-05-09T20:16:58.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.724+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=T32FUUQX25GA tag=97B19VO68P6J type=write size=3455704 2024-05-09T20:16:58.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.724+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=YFYC5H2V7VIL type=del 2024-05-09T20:16:58.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.728+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=M3E7FRWBYEDB type=write size=744629 2024-05-09T20:16:58.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.732+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=T2AU8BMD9TRX type=del 2024-05-09T20:16:58.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.736+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=RVT4T45AFSRU type=del size=902024 2024-05-09T20:16:58.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.740+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~YLTUPUGAZ0QX.0 size=1048576 2024-05-09T20:16:58.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.740+0000 7fdc882e58c0 1 < canceled part key=_multipart_54R9XPO62J0M.2~YLTUPUGAZ0QX.1 size=1048576 2024-05-09T20:16:58.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.744+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~YLTUPUGAZ0QX.2 size=818274 2024-05-09T20:16:58.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.748+0000 7fdc882e58c0 1 < canceled multipart upload key=54R9XPO62J0M upload=YLTUPUGAZ0QX tag=PHO2AE54XZZF type=write size=3964002 2024-05-09T20:16:58.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.752+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=GTH7HBWJLJM3 type=del size=3544319 2024-05-09T20:16:58.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.756+0000 7fdc882e58c0 1 < completed write operation key=FNB9B18LIA8L tag=U3TUSOWU3VUF type=write size=257614 2024-05-09T20:16:58.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.760+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=KZY8Z7YFVPI4 type=del size=3291180 2024-05-09T20:16:58.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.764+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~KGOKGMF8F4YP.0 size=1048576 2024-05-09T20:16:58.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.764+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~KGOKGMF8F4YP.1 size=427291 2024-05-09T20:16:58.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.772+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=KGOKGMF8F4YP tag=S5K1SP12Y444 type=write size=2524443 2024-05-09T20:16:58.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.776+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=T17UTX4JRETO type=del size=3760982 2024-05-09T20:16:58.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.780+0000 7fdc882e58c0 1 < suggested update operation key=T6T3IOW9VPA0 tag=0NMCW43XUOAP type=del size=2256215 2024-05-09T20:16:58.779 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.788+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~S2VM6L92MBKW.0 size=191067 2024-05-09T20:16:58.779 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.788+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=S2VM6L92MBKW tag=GAN0K5KI7ZL7 type=write size=1239643 2024-05-09T20:16:58.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.792+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~1JWG3W2Z5QLN.0 size=1048576 2024-05-09T20:16:58.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.792+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~1JWG3W2Z5QLN.1 size=1048576 2024-05-09T20:16:58.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.796+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~1JWG3W2Z5QLN.2 size=406556 2024-05-09T20:16:58.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.800+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=1JWG3W2Z5QLN tag=IR220Y25O4RG type=write size=3552284 2024-05-09T20:16:58.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.808+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~LBDFW5PNW3LM.0 size=1048576 2024-05-09T20:16:58.801 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.808+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~LBDFW5PNW3LM.1 size=1023415 2024-05-09T20:16:58.801 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.808+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=LBDFW5PNW3LM tag=KHATTPX3PTJJ type=write size=3120567 2024-05-09T20:16:58.802 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.812+0000 7fdc882e58c0 1 < canceled part key=_multipart_IZPPFLWVPRKO.2~SAW83TQQFIJ5.0 size=911752 2024-05-09T20:16:58.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.816+0000 7fdc882e58c0 1 < canceled multipart upload key=IZPPFLWVPRKO upload=SAW83TQQFIJ5 tag=OWCL8BNW9WAX type=write size=1960328 2024-05-09T20:16:58.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.820+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~OMBVRVS3FIYN.0 size=1048576 2024-05-09T20:16:58.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.824+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~OMBVRVS3FIYN.1 size=116088 2024-05-09T20:16:58.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.824+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=OMBVRVS3FIYN tag=JL63E7CQPQ62 type=write size=2213240 2024-05-09T20:16:58.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.828+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~DONVT7QIU50S.0 size=1048576 2024-05-09T20:16:58.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.828+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~DONVT7QIU50S.1 size=887838 2024-05-09T20:16:58.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.836+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=DONVT7QIU50S tag=K31JQE4KF9UK type=write size=2984990 2024-05-09T20:16:58.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.840+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~ZI2OAV4SRCX6.0 size=1048576 2024-05-09T20:16:58.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.844+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~ZI2OAV4SRCX6.1 size=1048576 2024-05-09T20:16:58.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.848+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~ZI2OAV4SRCX6.2 size=229359 2024-05-09T20:16:58.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.848+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=ZI2OAV4SRCX6 tag=CD8198RCQMMC type=write size=3375087 2024-05-09T20:16:58.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.848+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~YE67HL7X6XX7.0 size=1048576 2024-05-09T20:16:58.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.852+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~YE67HL7X6XX7.1 size=1048576 2024-05-09T20:16:58.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.856+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~YE67HL7X6XX7.2 size=834944 2024-05-09T20:16:58.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.860+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=YE67HL7X6XX7 tag=GHR99XDYJSY7 type=write size=3980672 2024-05-09T20:16:58.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.864+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=6LHL19SS7UZO type=del size=2823130 2024-05-09T20:16:58.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.868+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=24DFQCZPCZBD type=del 2024-05-09T20:16:58.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.868+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=CWJUH9T6Y3L5 type=del size=3079466 2024-05-09T20:16:58.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.872+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=NE0V85BNLOXS type=del 2024-05-09T20:16:58.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.876+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=M7XYTSRDA6X3 type=del size=170713 2024-05-09T20:16:58.870 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.880+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~DM0OLOBWCZQF.0 size=378597 2024-05-09T20:16:58.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.884+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=DM0OLOBWCZQF tag=BOTTWG86Z14W type=write size=1427173 2024-05-09T20:16:58.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.892+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~H0ZSY7ZABDAV.0 size=151166 2024-05-09T20:16:58.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.892+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=H0ZSY7ZABDAV tag=CL4EUEP3MJY2 type=write size=1199742 2024-05-09T20:16:58.886 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.892+0000 7fdc882e58c0 1 < canceled operation key=NYHR6GTPQYZS tag=4T7I7IJVU2L8 type=del size=2000119 2024-05-09T20:16:58.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.896+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=4IIVOQKJR4R3 type=del size=3042490 2024-05-09T20:16:58.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.900+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=1B7E8QHROUQK type=del 2024-05-09T20:16:58.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.904+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=A1CJDTXT9H4B type=del size=2120913 2024-05-09T20:16:58.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.908+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~T32FUUQX25GA.0 size=1048576 2024-05-09T20:16:58.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.908+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~T32FUUQX25GA.1 size=1048576 2024-05-09T20:16:58.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.908+0000 7fdc882e58c0 1 < canceled part key=_multipart_QM53IENH9CX4.2~T32FUUQX25GA.2 size=309976 2024-05-09T20:16:58.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.916+0000 7fdc882e58c0 1 < canceled multipart upload key=QM53IENH9CX4 upload=T32FUUQX25GA tag=97B19VO68P6J type=write size=3455704 2024-05-09T20:16:58.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.920+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~F5MK22JLBVQH.0 size=1048576 2024-05-09T20:16:58.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.924+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~F5MK22JLBVQH.1 size=1048576 2024-05-09T20:16:58.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.928+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~F5MK22JLBVQH.2 size=410203 2024-05-09T20:16:58.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.928+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=F5MK22JLBVQH tag=27IQEZJVUC3L type=write size=3555931 2024-05-09T20:16:58.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.932+0000 7fdc882e58c0 1 < suggested update operation key=NYHR6GTPQYZS tag=M3E7FRWBYEDB type=write size=744629 2024-05-09T20:16:58.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.936+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=AZ1Z2XZQ6JWC type=write size=451993 2024-05-09T20:16:58.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.940+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=RVT4T45AFSRU type=del 2024-05-09T20:16:58.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.944+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~9DMDVFZOXBFT.0 size=1048576 2024-05-09T20:16:58.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.948+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~9DMDVFZOXBFT.1 size=1048576 2024-05-09T20:16:58.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.952+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~9DMDVFZOXBFT.2 size=750843 2024-05-09T20:16:58.943 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.952+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=9DMDVFZOXBFT tag=BO9W53YJ1QLV type=write size=3896571 2024-05-09T20:16:58.946 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.956+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=GTH7HBWJLJM3 type=del 2024-05-09T20:16:58.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.960+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=6T57IXFX92AR type=del size=51392 2024-05-09T20:16:58.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.960+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=KZY8Z7YFVPI4 type=del 2024-05-09T20:16:58.958 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.964+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=H2VHR2CA3OZN type=del size=926312 2024-05-09T20:16:58.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.968+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=T17UTX4JRETO type=del 2024-05-09T20:16:58.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.972+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=MP2OCN14T5SB type=write size=585059 2024-05-09T20:16:58.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.976+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~S2VM6L92MBKW.0 size=191067 2024-05-09T20:16:58.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.984+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=S2VM6L92MBKW tag=GAN0K5KI7ZL7 type=write size=1239643 2024-05-09T20:16:58.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.988+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=KMNSWA82IB5T type=del size=3313261 2024-05-09T20:16:58.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.992+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~LBDFW5PNW3LM.0 size=1048576 2024-05-09T20:16:58.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:58.992+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~LBDFW5PNW3LM.1 size=1023415 2024-05-09T20:16:58.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.000+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=LBDFW5PNW3LM tag=KHATTPX3PTJJ type=write size=3120567 2024-05-09T20:16:58.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.004+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=C9VDIZEMQZVL type=write size=918575 2024-05-09T20:16:58.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.004+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~OMBVRVS3FIYN.0 size=1048576 2024-05-09T20:16:59.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.008+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~OMBVRVS3FIYN.1 size=116088 2024-05-09T20:16:59.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.016+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=OMBVRVS3FIYN tag=JL63E7CQPQ62 type=write size=2213240 2024-05-09T20:16:59.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.016+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=DR7H8FK9YTYB type=del size=2581014 2024-05-09T20:16:59.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.020+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~ZI2OAV4SRCX6.0 size=1048576 2024-05-09T20:16:59.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.024+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~ZI2OAV4SRCX6.1 size=1048576 2024-05-09T20:16:59.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.024+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~ZI2OAV4SRCX6.2 size=229359 2024-05-09T20:16:59.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.032+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=ZI2OAV4SRCX6 tag=CD8198RCQMMC type=write size=3375087 2024-05-09T20:16:59.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.036+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=LG8881LLARLY type=del size=2582358 2024-05-09T20:16:59.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.040+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=6LHL19SS7UZO type=del 2024-05-09T20:16:59.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.044+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~Y1CDB5W4U58M.0 size=827351 2024-05-09T20:16:59.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.044+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=Y1CDB5W4U58M tag=TRC30HUN866S type=write size=1875927 2024-05-09T20:16:59.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.048+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=CWJUH9T6Y3L5 type=del 2024-05-09T20:16:59.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.052+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=O3ASMN3Z04EE type=del size=2706205 2024-05-09T20:16:59.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.056+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=M7XYTSRDA6X3 type=del 2024-05-09T20:16:59.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.060+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=4138LD3CE3GA type=del size=4184340 2024-05-09T20:16:59.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.064+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~H0ZSY7ZABDAV.0 size=151166 2024-05-09T20:16:59.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.072+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=H0ZSY7ZABDAV tag=CL4EUEP3MJY2 type=write size=1199742 2024-05-09T20:16:59.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.076+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~C1U3JOE80N44.0 size=1048576 2024-05-09T20:16:59.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.080+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~C1U3JOE80N44.1 size=61877 2024-05-09T20:16:59.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.080+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=C1U3JOE80N44 tag=PW7S7COOIUPH type=write size=2159029 2024-05-09T20:16:59.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.084+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=4IIVOQKJR4R3 type=del 2024-05-09T20:16:59.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.088+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=EG9W94L5QNO4 type=del size=2097330 2024-05-09T20:16:59.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.088+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=A1CJDTXT9H4B type=del 2024-05-09T20:16:59.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.092+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=AUVH619DNSK7 type=del size=3615574 2024-05-09T20:16:59.088 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.096+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~F5MK22JLBVQH.0 size=1048576 2024-05-09T20:16:59.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.100+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~F5MK22JLBVQH.1 size=1048576 2024-05-09T20:16:59.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.104+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~F5MK22JLBVQH.2 size=410203 2024-05-09T20:16:59.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.108+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=F5MK22JLBVQH tag=27IQEZJVUC3L type=write size=3555931 2024-05-09T20:16:59.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.112+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=9Z1ILKHDHOO6 type=del size=4057118 2024-05-09T20:16:59.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.116+0000 7fdc882e58c0 1 < completed write operation key=NBAECV9L85UK tag=AZ1Z2XZQ6JWC type=write size=451993 2024-05-09T20:16:59.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.120+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=WWJYAOPGY3DW type=del size=659886 2024-05-09T20:16:59.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.120+0000 7fdc882e58c0 1 < canceled part key=_multipart_AJANS5ADJO5B.2~9DMDVFZOXBFT.0 size=1048576 2024-05-09T20:16:59.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.124+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~9DMDVFZOXBFT.1 size=1048576 2024-05-09T20:16:59.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.124+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~9DMDVFZOXBFT.2 size=750843 2024-05-09T20:16:59.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.132+0000 7fdc882e58c0 1 < canceled multipart upload key=AJANS5ADJO5B upload=9DMDVFZOXBFT tag=BO9W53YJ1QLV type=write size=3896571 2024-05-09T20:16:59.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.132+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=33UIG44IHY0N type=del size=4081582 2024-05-09T20:16:59.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.136+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=6T57IXFX92AR type=del 2024-05-09T20:16:59.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.140+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=GL944NI060XA type=del size=3908560 2024-05-09T20:16:59.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.144+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=H2VHR2CA3OZN type=del 2024-05-09T20:16:59.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.148+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=5P6SERQJH2M4 type=del size=384717 2024-05-09T20:16:59.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.152+0000 7fdc882e58c0 1 < completed write operation key=HPK2T0629B1P tag=MP2OCN14T5SB type=write size=585059 2024-05-09T20:16:59.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.152+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=4EK1JLXP21E8 type=del size=4008520 2024-05-09T20:16:59.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.160+0000 7fdc882e58c0 1 < suggested update operation key=O07UHAIJZF0R tag=KMNSWA82IB5T type=del size=3313261 2024-05-09T20:16:59.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.164+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=2F81UYCM8J97 type=del size=1769367 2024-05-09T20:16:59.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.172+0000 7fdc882e58c0 1 < suggested update operation key=346HV1XQKCOZ tag=C9VDIZEMQZVL type=write size=918575 2024-05-09T20:16:59.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.180+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~YM8EX6VCL6GQ.0 size=1048576 2024-05-09T20:16:59.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.180+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~YM8EX6VCL6GQ.1 size=1048576 2024-05-09T20:16:59.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.184+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~YM8EX6VCL6GQ.2 size=5406 2024-05-09T20:16:59.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.184+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=YM8EX6VCL6GQ tag=6O6MLINTBCZU type=write size=3151134 2024-05-09T20:16:59.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.192+0000 7fdc882e58c0 1 < suggested remove operation key=5NEOR8OOFAFZ tag=DR7H8FK9YTYB type=del 2024-05-09T20:16:59.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.200+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~7TRL1YH62C13.0 size=1048576 2024-05-09T20:16:59.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.204+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~7TRL1YH62C13.1 size=1048576 2024-05-09T20:16:59.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.208+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~7TRL1YH62C13.2 size=173296 2024-05-09T20:16:59.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.208+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=7TRL1YH62C13 tag=9CB00WXBFUPE type=write size=3319024 2024-05-09T20:16:59.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.212+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=LG8881LLARLY type=del 2024-05-09T20:16:59.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.216+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=1E57MTTSECHW type=del size=2035312 2024-05-09T20:16:59.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.216+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~Y1CDB5W4U58M.0 size=827351 2024-05-09T20:16:59.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.224+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=Y1CDB5W4U58M tag=TRC30HUN866S type=write size=1875927 2024-05-09T20:16:59.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.228+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=T2HHRMXLJRHC type=del size=1364692 2024-05-09T20:16:59.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.232+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=O3ASMN3Z04EE type=del 2024-05-09T20:16:59.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.232+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=8O961KJNFPGT type=write size=140657 2024-05-09T20:16:59.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.236+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=4138LD3CE3GA type=del 2024-05-09T20:16:59.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.240+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=OM7VVNIGBY9Z type=del size=1486213 2024-05-09T20:16:59.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.244+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~C1U3JOE80N44.0 size=1048576 2024-05-09T20:16:59.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.248+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~C1U3JOE80N44.1 size=61877 2024-05-09T20:16:59.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.252+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=C1U3JOE80N44 tag=PW7S7COOIUPH type=write size=2159029 2024-05-09T20:16:59.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.256+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=YKME102Z5V3X type=write size=81220 2024-05-09T20:16:59.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.260+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=EG9W94L5QNO4 type=del 2024-05-09T20:16:59.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.268+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~I7LSQ8R80OA5.0 size=390865 2024-05-09T20:16:59.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.268+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=I7LSQ8R80OA5 tag=PWW4TVYANEFA type=write size=1439441 2024-05-09T20:16:59.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.272+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=AUVH619DNSK7 type=del 2024-05-09T20:16:59.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.276+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=OQV8MEQ43HNM type=del size=2949808 2024-05-09T20:16:59.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.280+0000 7fdc882e58c0 1 < canceled operation key=P2DNA4DGI54P tag=9Z1ILKHDHOO6 type=del size=4057118 2024-05-09T20:16:59.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.284+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~ECZEVQ9IQLVK.0 size=1048576 2024-05-09T20:16:59.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.288+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~ECZEVQ9IQLVK.1 size=965509 2024-05-09T20:16:59.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.288+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=ECZEVQ9IQLVK tag=TIRT12KP98WH type=write size=3062661 2024-05-09T20:16:59.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.292+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=WWJYAOPGY3DW type=del 2024-05-09T20:16:59.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.296+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=9POFO25WAI75 type=write size=770698 2024-05-09T20:16:59.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.304+0000 7fdc882e58c0 1 < suggested remove operation key=8FFZZDURUWM1 tag=33UIG44IHY0N type=del 2024-05-09T20:16:59.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.308+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=8PWUR1WPJHQA type=del size=2174453 2024-05-09T20:16:59.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.316+0000 7fdc882e58c0 1 < suggested remove operation key=OGVA2778MJWU tag=GL944NI060XA type=del 2024-05-09T20:16:59.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.320+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=L5F6GC355OZK type=del size=2948322 2024-05-09T20:16:59.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.328+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=5P6SERQJH2M4 type=del size=384717 2024-05-09T20:16:59.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.332+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=9CDLEJ4DEV4G type=write size=535 2024-05-09T20:16:59.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.340+0000 7fdc882e58c0 1 < suggested update operation key=FNB9B18LIA8L tag=4EK1JLXP21E8 type=del size=4008520 2024-05-09T20:16:59.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.344+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~FR2SRCFO8FCC.0 size=1048576 2024-05-09T20:16:59.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.348+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~FR2SRCFO8FCC.1 size=1048576 2024-05-09T20:16:59.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.352+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~FR2SRCFO8FCC.2 size=71136 2024-05-09T20:16:59.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.352+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=FR2SRCFO8FCC tag=8TOV6XH0JEHI type=write size=3216864 2024-05-09T20:16:59.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.352+0000 7fdc882e58c0 -1 < no bi entry to suggest for operation key=5NEOR8OOFAFZ tag=2F81UYCM8J97 type=del size=1769367: (2) No such file or directory 2024-05-09T20:16:59.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.356+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=AC5LB8E172DG type=del size=3124754 2024-05-09T20:16:59.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.360+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~YM8EX6VCL6GQ.0 size=1048576 2024-05-09T20:16:59.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.364+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~YM8EX6VCL6GQ.1 size=1048576 2024-05-09T20:16:59.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.368+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~YM8EX6VCL6GQ.2 size=5406 2024-05-09T20:16:59.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.372+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=YM8EX6VCL6GQ tag=6O6MLINTBCZU type=write size=3151134 2024-05-09T20:16:59.371 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.380+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~DQ6OFNX49I3Y.0 size=1026816 2024-05-09T20:16:59.371 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.380+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=DQ6OFNX49I3Y tag=W8EQKEQKYLFM type=write size=2075392 2024-05-09T20:16:59.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.380+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~7TRL1YH62C13.0 size=1048576 2024-05-09T20:16:59.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.384+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~7TRL1YH62C13.1 size=1048576 2024-05-09T20:16:59.377 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.384+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~7TRL1YH62C13.2 size=173296 2024-05-09T20:16:59.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.392+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=7TRL1YH62C13 tag=9CB00WXBFUPE type=write size=3319024 2024-05-09T20:16:59.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.396+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=O3GDFFL6GQI9 type=del size=3053878 2024-05-09T20:16:59.392 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.400+0000 7fdc882e58c0 1 < suggested remove operation key=O07UHAIJZF0R tag=1E57MTTSECHW type=del 2024-05-09T20:16:59.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.408+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~WB710FH5HV3W.0 size=625927 2024-05-09T20:16:59.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.408+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=WB710FH5HV3W tag=GIOASXBGERHL type=write size=1674503 2024-05-09T20:16:59.405 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.412+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=T2HHRMXLJRHC type=del 2024-05-09T20:16:59.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.420+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~YDV69ZI8ZNOP.0 size=1048576 2024-05-09T20:16:59.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.420+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~YDV69ZI8ZNOP.1 size=726341 2024-05-09T20:16:59.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.420+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=YDV69ZI8ZNOP tag=X0MULZGAQN8C type=write size=2823493 2024-05-09T20:16:59.416 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.424+0000 7fdc882e58c0 -1 < failed to complete operation key=8FFZZDURUWM1 tag=8O961KJNFPGT type=write size=140657: (22) Invalid argument 2024-05-09T20:16:59.419 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.428+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=QY3EHGE0S4JS type=del size=1711836 2024-05-09T20:16:59.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.432+0000 7fdc882e58c0 1 < suggested update operation key=G0DX1F257RLD tag=OM7VVNIGBY9Z type=del size=1486213 2024-05-09T20:16:59.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.440+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~P1L8O1YTYE7V.0 size=1048576 2024-05-09T20:16:59.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.440+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~P1L8O1YTYE7V.1 size=1048576 2024-05-09T20:16:59.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.444+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~P1L8O1YTYE7V.2 size=195765 2024-05-09T20:16:59.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.444+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=P1L8O1YTYE7V tag=YVE87SCB7EWH type=write size=3341493 2024-05-09T20:16:59.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.448+0000 7fdc882e58c0 1 < completed write operation key=346HV1XQKCOZ tag=YKME102Z5V3X type=write size=81220 2024-05-09T20:16:59.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.452+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~LWN507BDR1RI.0 size=1048576 2024-05-09T20:16:59.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.456+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~LWN507BDR1RI.1 size=78383 2024-05-09T20:16:59.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.456+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=LWN507BDR1RI tag=5244I6OGR7LW type=write size=2175535 2024-05-09T20:16:59.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.460+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~I7LSQ8R80OA5.0 size=390865 2024-05-09T20:16:59.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.464+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=I7LSQ8R80OA5 tag=PWW4TVYANEFA type=write size=1439441 2024-05-09T20:16:59.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.468+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=BA0HIX0ORK3T type=del size=1418886 2024-05-09T20:16:59.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.472+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=OQV8MEQ43HNM type=del 2024-05-09T20:16:59.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.476+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=QXIMV3NBO3O7 type=write size=7728 2024-05-09T20:16:59.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.480+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~ECZEVQ9IQLVK.0 size=1048576 2024-05-09T20:16:59.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.480+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~ECZEVQ9IQLVK.1 size=965509 2024-05-09T20:16:59.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.484+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=ECZEVQ9IQLVK tag=TIRT12KP98WH type=write size=3062661 2024-05-09T20:16:59.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.488+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=ZN3QTV0TCHWH type=del size=286161 2024-05-09T20:16:59.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.492+0000 7fdc882e58c0 -1 < failed to complete operation key=EK78CNKUHNKO tag=9POFO25WAI75 type=write size=770698: (22) Invalid argument 2024-05-09T20:16:59.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.496+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=S4Y24U9MSHQX type=del size=1873273 2024-05-09T20:16:59.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.496+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=8PWUR1WPJHQA type=del 2024-05-09T20:16:59.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.500+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=UZP8NTSQG825 type=del size=1473858 2024-05-09T20:16:59.497 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.504+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=L5F6GC355OZK type=del 2024-05-09T20:16:59.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.508+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=KYS06NF4I7QP type=del size=2274286 2024-05-09T20:16:59.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.512+0000 7fdc882e58c0 1 < completed write operation key=TQ5ZHATLNVPP tag=9CDLEJ4DEV4G type=write size=535 2024-05-09T20:16:59.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.516+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=OL7CN2MJLW8I type=write size=186222 2024-05-09T20:16:59.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.520+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~FR2SRCFO8FCC.0 size=1048576 2024-05-09T20:16:59.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.520+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~FR2SRCFO8FCC.1 size=1048576 2024-05-09T20:16:59.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.524+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~FR2SRCFO8FCC.2 size=71136 2024-05-09T20:16:59.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.528+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=FR2SRCFO8FCC tag=8TOV6XH0JEHI type=write size=3216864 2024-05-09T20:16:59.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.536+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~ND5SN6H1E9QZ.0 size=1048576 2024-05-09T20:16:59.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.540+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~ND5SN6H1E9QZ.1 size=544341 2024-05-09T20:16:59.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.540+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=ND5SN6H1E9QZ tag=HDRU2E51EKUI type=write size=2641493 2024-05-09T20:16:59.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.544+0000 7fdc882e58c0 1 < canceled operation key=CNV7NHP27JZ7 tag=AC5LB8E172DG type=del size=3124754 2024-05-09T20:16:59.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.544+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=717G73ZNAZRC type=del size=2070384 2024-05-09T20:16:59.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.548+0000 7fdc882e58c0 1 < canceled part key=_multipart_5NEOR8OOFAFZ.2~DQ6OFNX49I3Y.0 size=1026816 2024-05-09T20:16:59.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.552+0000 7fdc882e58c0 1 < canceled multipart upload key=5NEOR8OOFAFZ upload=DQ6OFNX49I3Y tag=W8EQKEQKYLFM type=write size=2075392 2024-05-09T20:16:59.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.560+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~P21ZNK0TSG5R.0 size=1048576 2024-05-09T20:16:59.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.564+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~P21ZNK0TSG5R.1 size=135850 2024-05-09T20:16:59.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.564+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=P21ZNK0TSG5R tag=0007A4210OFF type=write size=2233002 2024-05-09T20:16:59.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.568+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=O3GDFFL6GQI9 type=del 2024-05-09T20:16:59.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.572+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~4DA1DF0BDM5J.0 size=583401 2024-05-09T20:16:59.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.572+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=4DA1DF0BDM5J tag=V5ZLYRPAZKH6 type=write size=1631977 2024-05-09T20:16:59.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.576+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~WB710FH5HV3W.0 size=625927 2024-05-09T20:16:59.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.584+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=WB710FH5HV3W tag=GIOASXBGERHL type=write size=1674503 2024-05-09T20:16:59.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.584+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=JACS5OI551JU type=del size=921145 2024-05-09T20:16:59.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.588+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~YDV69ZI8ZNOP.0 size=1048576 2024-05-09T20:16:59.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.592+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~YDV69ZI8ZNOP.1 size=726341 2024-05-09T20:16:59.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.596+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=YDV69ZI8ZNOP tag=X0MULZGAQN8C type=write size=2823493 2024-05-09T20:16:59.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.600+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=X2KDGSL93G9P type=write size=568791 2024-05-09T20:16:59.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.604+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=QY3EHGE0S4JS type=del 2024-05-09T20:16:59.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.608+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~733XPMO3OVHM.0 size=1048576 2024-05-09T20:16:59.603 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~733XPMO3OVHM.1 size=468630 2024-05-09T20:16:59.603 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.612+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=733XPMO3OVHM tag=8UXU6XHC3JA4 type=write size=2565782 2024-05-09T20:16:59.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.612+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~P1L8O1YTYE7V.0 size=1048576 2024-05-09T20:16:59.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.612+0000 7fdc882e58c0 1 < canceled part key=_multipart_AJANS5ADJO5B.2~P1L8O1YTYE7V.1 size=1048576 2024-05-09T20:16:59.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.616+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~P1L8O1YTYE7V.2 size=195765 2024-05-09T20:16:59.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.620+0000 7fdc882e58c0 1 < canceled multipart upload key=AJANS5ADJO5B upload=P1L8O1YTYE7V tag=YVE87SCB7EWH type=write size=3341493 2024-05-09T20:16:59.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.628+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~XPG3EED53EGQ.0 size=96487 2024-05-09T20:16:59.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.628+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=XPG3EED53EGQ tag=6T4X85GPE7Z2 type=write size=1145063 2024-05-09T20:16:59.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.632+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~LWN507BDR1RI.0 size=1048576 2024-05-09T20:16:59.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.632+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~LWN507BDR1RI.1 size=78383 2024-05-09T20:16:59.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.640+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=LWN507BDR1RI tag=5244I6OGR7LW type=write size=2175535 2024-05-09T20:16:59.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.644+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=CBNK7BCS11T6 type=del size=2504276 2024-05-09T20:16:59.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.648+0000 7fdc882e58c0 1 < suggested remove operation key=8FFZZDURUWM1 tag=BA0HIX0ORK3T type=del 2024-05-09T20:16:59.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.656+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~TQGVPS0KLNY6.0 size=224300 2024-05-09T20:16:59.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.656+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=TQGVPS0KLNY6 tag=GAZDQRSOR6L4 type=write size=1272876 2024-05-09T20:16:59.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.656+0000 7fdc882e58c0 1 < completed write operation key=TQ5ZHATLNVPP tag=QXIMV3NBO3O7 type=write size=7728 2024-05-09T20:16:59.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.660+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=BCFQAMQGGWEP type=del size=1083966 2024-05-09T20:16:59.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.664+0000 7fdc882e58c0 1 < canceled operation key=NNGWUSCETMH4 tag=ZN3QTV0TCHWH type=del size=286161 2024-05-09T20:16:59.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.668+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~24FNQUAXWJJS.0 size=1048576 2024-05-09T20:16:59.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.672+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~24FNQUAXWJJS.1 size=1048576 2024-05-09T20:16:59.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.676+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~24FNQUAXWJJS.2 size=884141 2024-05-09T20:16:59.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.676+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=24FNQUAXWJJS tag=NNCPPXQSI1FU type=write size=4029869 2024-05-09T20:16:59.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.676+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=S4Y24U9MSHQX type=del 2024-05-09T20:16:59.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.684+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~KGKUR7ZC8HGA.0 size=1048576 2024-05-09T20:16:59.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.684+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~KGKUR7ZC8HGA.1 size=1048576 2024-05-09T20:16:59.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.688+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~KGKUR7ZC8HGA.2 size=443938 2024-05-09T20:16:59.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.688+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=KGKUR7ZC8HGA tag=XHANP8V4JPE6 type=write size=3589666 2024-05-09T20:16:59.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.692+0000 7fdc882e58c0 1 < canceled operation key=QM53IENH9CX4 tag=UZP8NTSQG825 type=del size=1473858 2024-05-09T20:16:59.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.692+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=A3NEGKSG6QZO type=write size=922206 2024-05-09T20:16:59.695 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.700+0000 7fdc882e58c0 1 < suggested update operation key=54R9XPO62J0M tag=KYS06NF4I7QP type=del size=2274286 2024-05-09T20:16:59.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.708+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~68UM7CO5TC5B.0 size=1048576 2024-05-09T20:16:59.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~68UM7CO5TC5B.1 size=1048576 2024-05-09T20:16:59.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~68UM7CO5TC5B.2 size=65617 2024-05-09T20:16:59.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.712+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=68UM7CO5TC5B tag=HHCD24G443Q0 type=write size=3211345 2024-05-09T20:16:59.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.716+0000 7fdc882e58c0 1 < canceled operation key=JS90W6PIREMD tag=OL7CN2MJLW8I type=write size=186222 2024-05-09T20:16:59.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.724+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~NMHQ6UJINL55.0 size=1048576 2024-05-09T20:16:59.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.728+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~NMHQ6UJINL55.1 size=1048576 2024-05-09T20:16:59.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.732+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~NMHQ6UJINL55.2 size=1000319 2024-05-09T20:16:59.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.732+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=NMHQ6UJINL55 tag=A8PH56JW6D1O type=write size=4146047 2024-05-09T20:16:59.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.732+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~ND5SN6H1E9QZ.0 size=1048576 2024-05-09T20:16:59.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.736+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~ND5SN6H1E9QZ.1 size=544341 2024-05-09T20:16:59.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.740+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=ND5SN6H1E9QZ tag=HDRU2E51EKUI type=write size=2641493 2024-05-09T20:16:59.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.744+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=7HTJ311607FS type=del size=497678 2024-05-09T20:16:59.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.748+0000 7fdc882e58c0 -1 < failed to complete operation key=54R9XPO62J0M tag=717G73ZNAZRC type=del size=2070384: (22) Invalid argument 2024-05-09T20:16:59.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.752+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~MM4W4XK3PDF3.0 size=816329 2024-05-09T20:16:59.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.752+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=MM4W4XK3PDF3 tag=18HIIVUIW3R3 type=write size=1864905 2024-05-09T20:16:59.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.756+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~P21ZNK0TSG5R.0 size=1048576 2024-05-09T20:16:59.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.760+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~P21ZNK0TSG5R.1 size=135850 2024-05-09T20:16:59.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.768+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=P21ZNK0TSG5R tag=0007A4210OFF type=write size=2233002 2024-05-09T20:16:59.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.772+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~V0YFPN7QNTZM.0 size=115593 2024-05-09T20:16:59.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.772+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=V0YFPN7QNTZM tag=XN3E2RJR8MHA type=write size=1164169 2024-05-09T20:16:59.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.776+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~4DA1DF0BDM5J.0 size=583401 2024-05-09T20:16:59.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.784+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=4DA1DF0BDM5J tag=V5ZLYRPAZKH6 type=write size=1631977 2024-05-09T20:16:59.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.788+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~PBHPGT5JG3R7.0 size=1048576 2024-05-09T20:16:59.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.792+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~PBHPGT5JG3R7.1 size=125531 2024-05-09T20:16:59.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.792+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=PBHPGT5JG3R7 tag=GTLT6Q2VGGQ5 type=write size=2222683 2024-05-09T20:16:59.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.800+0000 7fdc882e58c0 1 < suggested update operation key=CNV7NHP27JZ7 tag=JACS5OI551JU type=del size=921145 2024-05-09T20:16:59.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.800+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=NKUY1WTK9WYJ type=del size=133148 2024-05-09T20:16:59.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.804+0000 7fdc882e58c0 1 < completed write operation key=BS0T1OQOBS67 tag=X2KDGSL93G9P type=write size=568791 2024-05-09T20:16:59.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.808+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=UADX36W2XWO3 type=del size=2210867 2024-05-09T20:16:59.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.812+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~733XPMO3OVHM.0 size=1048576 2024-05-09T20:16:59.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.816+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~733XPMO3OVHM.1 size=468630 2024-05-09T20:16:59.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.820+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=733XPMO3OVHM tag=8UXU6XHC3JA4 type=write size=2565782 2024-05-09T20:16:59.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.828+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~4ZQVT1H1JNTU.0 size=1048576 2024-05-09T20:16:59.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.832+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~4ZQVT1H1JNTU.1 size=130796 2024-05-09T20:16:59.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.832+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=4ZQVT1H1JNTU tag=PD6NI0BJOHHI type=write size=2227948 2024-05-09T20:16:59.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.836+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~XPG3EED53EGQ.0 size=96487 2024-05-09T20:16:59.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.840+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=XPG3EED53EGQ tag=6T4X85GPE7Z2 type=write size=1145063 2024-05-09T20:16:59.839 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.848+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~NTYDF5V6MQRK.0 size=1048576 2024-05-09T20:16:59.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.848+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~NTYDF5V6MQRK.1 size=1048576 2024-05-09T20:16:59.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.852+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~NTYDF5V6MQRK.2 size=835657 2024-05-09T20:16:59.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.852+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=NTYDF5V6MQRK tag=2E266OW1QWCY type=write size=3981385 2024-05-09T20:16:59.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.856+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=CBNK7BCS11T6 type=del 2024-05-09T20:16:59.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.860+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=AL53F48PJJU3 type=del size=4019634 2024-05-09T20:16:59.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.864+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~TQGVPS0KLNY6.0 size=224300 2024-05-09T20:16:59.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.868+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=TQGVPS0KLNY6 tag=GAZDQRSOR6L4 type=write size=1272876 2024-05-09T20:16:59.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.872+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=7011EJT1D8H6 type=del size=4055700 2024-05-09T20:16:59.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.876+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=BCFQAMQGGWEP type=del 2024-05-09T20:16:59.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.884+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~SP1AM9WUQW97.0 size=789171 2024-05-09T20:16:59.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.884+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=SP1AM9WUQW97 tag=F4IDYZOVYSMN type=write size=1837747 2024-05-09T20:16:59.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.884+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~24FNQUAXWJJS.0 size=1048576 2024-05-09T20:16:59.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.888+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~24FNQUAXWJJS.1 size=1048576 2024-05-09T20:16:59.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.892+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~24FNQUAXWJJS.2 size=884141 2024-05-09T20:16:59.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.896+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=24FNQUAXWJJS tag=NNCPPXQSI1FU type=write size=4029869 2024-05-09T20:16:59.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.900+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=3Z04Y8Z0Z1P3 type=write size=311865 2024-05-09T20:16:59.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.904+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~KGKUR7ZC8HGA.0 size=1048576 2024-05-09T20:16:59.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.908+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~KGKUR7ZC8HGA.1 size=1048576 2024-05-09T20:16:59.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.912+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~KGKUR7ZC8HGA.2 size=443938 2024-05-09T20:16:59.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.916+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=KGKUR7ZC8HGA tag=XHANP8V4JPE6 type=write size=3589666 2024-05-09T20:16:59.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.924+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~VIOC67QWNVP2.0 size=1048576 2024-05-09T20:16:59.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.928+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~VIOC67QWNVP2.1 size=1048576 2024-05-09T20:16:59.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.928+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~VIOC67QWNVP2.2 size=720868 2024-05-09T20:16:59.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.928+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=VIOC67QWNVP2 tag=YKIJQV5Q2OG7 type=write size=3866596 2024-05-09T20:16:59.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.932+0000 7fdc882e58c0 1 < completed write operation key=SPXHCWZICCWY tag=A3NEGKSG6QZO type=write size=922206 2024-05-09T20:16:59.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.940+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~9WHQXCSRISCZ.0 size=635266 2024-05-09T20:16:59.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.940+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=9WHQXCSRISCZ tag=8AP3C7A1EAUV type=write size=1683842 2024-05-09T20:16:59.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.944+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~68UM7CO5TC5B.0 size=1048576 2024-05-09T20:16:59.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.944+0000 7fdc882e58c0 1 < canceled part key=_multipart_SPXHCWZICCWY.2~68UM7CO5TC5B.1 size=1048576 2024-05-09T20:16:59.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.944+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~68UM7CO5TC5B.2 size=65617 2024-05-09T20:16:59.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.952+0000 7fdc882e58c0 1 < canceled multipart upload key=SPXHCWZICCWY upload=68UM7CO5TC5B tag=HHCD24G443Q0 type=write size=3211345 2024-05-09T20:16:59.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.956+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=KWXPJ09PO04T type=del size=3372933 2024-05-09T20:16:59.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.960+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~NMHQ6UJINL55.0 size=1048576 2024-05-09T20:16:59.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.964+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~NMHQ6UJINL55.1 size=1048576 2024-05-09T20:16:59.957 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.964+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~NMHQ6UJINL55.2 size=1000319 2024-05-09T20:16:59.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.972+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=NMHQ6UJINL55 tag=A8PH56JW6D1O type=write size=4146047 2024-05-09T20:16:59.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~S8W6EG36TVL2.0 size=1048576 2024-05-09T20:16:59.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~S8W6EG36TVL2.1 size=1048576 2024-05-09T20:16:59.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.984+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~S8W6EG36TVL2.2 size=665578 2024-05-09T20:16:59.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.984+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=S8W6EG36TVL2 tag=JE2ARW6VK3GH type=write size=3811306 2024-05-09T20:16:59.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.988+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=7HTJ311607FS type=del 2024-05-09T20:16:59.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.992+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=9ZJDM4JJLUK7 type=del size=2988415 2024-05-09T20:16:59.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:16:59.996+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~MM4W4XK3PDF3.0 size=816329 2024-05-09T20:16:59.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.000+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=MM4W4XK3PDF3 tag=18HIIVUIW3R3 type=write size=1864905 2024-05-09T20:16:59.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.004+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=WFOYRU94WQ2L type=del size=1921361 2024-05-09T20:16:59.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.008+0000 7fdc882e58c0 1 < canceled part key=_multipart_54R9XPO62J0M.2~V0YFPN7QNTZM.0 size=115593 2024-05-09T20:17:00.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.012+0000 7fdc882e58c0 1 < canceled multipart upload key=54R9XPO62J0M upload=V0YFPN7QNTZM tag=XN3E2RJR8MHA type=write size=1164169 2024-05-09T20:17:00.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.020+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~P4ML1TMAFOZU.0 size=1048576 2024-05-09T20:17:00.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.024+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~P4ML1TMAFOZU.1 size=198534 2024-05-09T20:17:00.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.024+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=P4ML1TMAFOZU tag=I99U28NUAV4D type=write size=2295686 2024-05-09T20:17:00.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.024+0000 7fdc882e58c0 1 < canceled part key=_multipart_T6T3IOW9VPA0.2~PBHPGT5JG3R7.0 size=1048576 2024-05-09T20:17:00.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.028+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~PBHPGT5JG3R7.1 size=125531 2024-05-09T20:17:00.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.040+0000 7fdc882e58c0 1 < canceled multipart upload key=T6T3IOW9VPA0 upload=PBHPGT5JG3R7 tag=GTLT6Q2VGGQ5 type=write size=2222683 2024-05-09T20:17:00.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.044+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=Y6XJXVCK5R4S type=write size=936657 2024-05-09T20:17:00.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.048+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=NKUY1WTK9WYJ type=del 2024-05-09T20:17:00.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.048+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=LW9JRJSYXENR type=del size=730427 2024-05-09T20:17:00.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.056+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=UADX36W2XWO3 type=del size=2210867 2024-05-09T20:17:00.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.064+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~4CNQPLL7X5N5.0 size=419753 2024-05-09T20:17:00.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.064+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=4CNQPLL7X5N5 tag=WETH0PFPV58R type=write size=1468329 2024-05-09T20:17:00.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.068+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~4ZQVT1H1JNTU.0 size=1048576 2024-05-09T20:17:00.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.072+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~4ZQVT1H1JNTU.1 size=130796 2024-05-09T20:17:00.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.076+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=4ZQVT1H1JNTU tag=PD6NI0BJOHHI type=write size=2227948 2024-05-09T20:17:00.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.080+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=0NAZN8SA8OB4 type=del size=2499490 2024-05-09T20:17:00.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.084+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~NTYDF5V6MQRK.0 size=1048576 2024-05-09T20:17:00.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.088+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~NTYDF5V6MQRK.1 size=1048576 2024-05-09T20:17:00.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.088+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~NTYDF5V6MQRK.2 size=835657 2024-05-09T20:17:00.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.096+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=NTYDF5V6MQRK tag=2E266OW1QWCY type=write size=3981385 2024-05-09T20:17:00.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.100+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=0S7GGWIREG6N type=del size=296469 2024-05-09T20:17:00.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.100+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=AL53F48PJJU3 type=del 2024-05-09T20:17:00.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.108+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~77G33XV0SSQY.0 size=563244 2024-05-09T20:17:00.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.108+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=77G33XV0SSQY tag=ZA64M7CPXJO3 type=write size=1611820 2024-05-09T20:17:00.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.112+0000 7fdc882e58c0 1 < suggested update operation key=SPXHCWZICCWY tag=7011EJT1D8H6 type=del size=4055700 2024-05-09T20:17:00.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.116+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=6TNYX6M9NMUF type=del size=2948677 2024-05-09T20:17:00.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.120+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~SP1AM9WUQW97.0 size=789171 2024-05-09T20:17:00.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.124+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=SP1AM9WUQW97 tag=F4IDYZOVYSMN type=write size=1837747 2024-05-09T20:17:00.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.128+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=FZIU30O8LNEM type=del size=1222593 2024-05-09T20:17:00.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.132+0000 7fdc882e58c0 1 < completed write operation key=JS90W6PIREMD tag=3Z04Y8Z0Z1P3 type=write size=311865 2024-05-09T20:17:00.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.132+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=VXSXI865TQRJ type=write size=1003876 2024-05-09T20:17:00.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.136+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~VIOC67QWNVP2.0 size=1048576 2024-05-09T20:17:00.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.140+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~VIOC67QWNVP2.1 size=1048576 2024-05-09T20:17:00.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.144+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~VIOC67QWNVP2.2 size=720868 2024-05-09T20:17:00.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.148+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=VIOC67QWNVP2 tag=YKIJQV5Q2OG7 type=write size=3866596 2024-05-09T20:17:00.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.152+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=1B7PP1MXLYZR type=del size=537552 2024-05-09T20:17:00.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.152+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~9WHQXCSRISCZ.0 size=635266 2024-05-09T20:17:00.149 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.156+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=9WHQXCSRISCZ tag=8AP3C7A1EAUV type=write size=1683842 2024-05-09T20:17:00.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.160+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=1TCFNFO1C8CN type=del size=2864253 2024-05-09T20:17:00.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.160+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=KWXPJ09PO04T type=del 2024-05-09T20:17:00.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.164+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=VGNDV9AVDB7N type=del size=17797 2024-05-09T20:17:00.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.168+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~S8W6EG36TVL2.0 size=1048576 2024-05-09T20:17:00.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.168+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~S8W6EG36TVL2.1 size=1048576 2024-05-09T20:17:00.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.172+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~S8W6EG36TVL2.2 size=665578 2024-05-09T20:17:00.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.176+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=S8W6EG36TVL2 tag=JE2ARW6VK3GH type=write size=3811306 2024-05-09T20:17:00.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.176+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=7WGLEC2Z8JJ5 type=del size=193480 2024-05-09T20:17:00.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.180+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=9ZJDM4JJLUK7 type=del 2024-05-09T20:17:00.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.180+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=3NB7B439L62H type=del size=1302920 2024-05-09T20:17:00.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.184+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=WFOYRU94WQ2L type=del 2024-05-09T20:17:00.179 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.188+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=I3PG9T3XG6W6 type=del size=2500889 2024-05-09T20:17:00.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.188+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~P4ML1TMAFOZU.0 size=1048576 2024-05-09T20:17:00.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.192+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~P4ML1TMAFOZU.1 size=198534 2024-05-09T20:17:00.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.196+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=P4ML1TMAFOZU tag=I99U28NUAV4D type=write size=2295686 2024-05-09T20:17:00.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.200+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=0UOLLGYE1XU3 type=del size=3247614 2024-05-09T20:17:00.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.204+0000 7fdc882e58c0 1 < canceled operation key=QM53IENH9CX4 tag=Y6XJXVCK5R4S type=write size=936657 2024-05-09T20:17:00.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.208+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=LHQD98CLPPBK type=del size=2036586 2024-05-09T20:17:00.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.212+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=LW9JRJSYXENR type=del 2024-05-09T20:17:00.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.216+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~JKCSRCH0DE79.0 size=1048576 2024-05-09T20:17:00.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.220+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~JKCSRCH0DE79.1 size=1048576 2024-05-09T20:17:00.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.224+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~JKCSRCH0DE79.2 size=732187 2024-05-09T20:17:00.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.224+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=JKCSRCH0DE79 tag=6EIQA8N6X58P type=write size=3877915 2024-05-09T20:17:00.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.228+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~4CNQPLL7X5N5.0 size=419753 2024-05-09T20:17:00.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.232+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=4CNQPLL7X5N5 tag=WETH0PFPV58R type=write size=1468329 2024-05-09T20:17:00.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.240+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~T3ES05YY4XCX.0 size=137628 2024-05-09T20:17:00.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.240+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=T3ES05YY4XCX tag=U9NWCBBDZU25 type=write size=1186204 2024-05-09T20:17:00.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.244+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=0NAZN8SA8OB4 type=del 2024-05-09T20:17:00.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.248+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~IXG9N137NZI2.0 size=150235 2024-05-09T20:17:00.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.248+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=IXG9N137NZI2 tag=T78KCWYU4E2Q type=write size=1198811 2024-05-09T20:17:00.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.252+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=0S7GGWIREG6N type=del 2024-05-09T20:17:00.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.260+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~JJYYOHD6L7KT.0 size=1048576 2024-05-09T20:17:00.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.260+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~JJYYOHD6L7KT.1 size=208730 2024-05-09T20:17:00.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.260+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=JJYYOHD6L7KT tag=0YBWE201Y6EM type=write size=2305882 2024-05-09T20:17:00.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.264+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~77G33XV0SSQY.0 size=563244 2024-05-09T20:17:00.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.272+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=77G33XV0SSQY tag=ZA64M7CPXJO3 type=write size=1611820 2024-05-09T20:17:00.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.276+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=NCTBTOTC70KO type=write size=659996 2024-05-09T20:17:00.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.276+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=6TNYX6M9NMUF type=del 2024-05-09T20:17:00.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.280+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=FYT0CJB5FOOT type=del size=1401227 2024-05-09T20:17:00.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.284+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=FZIU30O8LNEM type=del 2024-05-09T20:17:00.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.288+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=UYOYMAXY7C6E type=del size=928062 2024-05-09T20:17:00.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.292+0000 7fdc882e58c0 1 < completed write operation key=BS0T1OQOBS67 tag=VXSXI865TQRJ type=write size=1003876 2024-05-09T20:17:00.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.296+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=2YCBRQLI35L1 type=write size=370315 2024-05-09T20:17:00.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.296+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=1B7PP1MXLYZR type=del 2024-05-09T20:17:00.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.304+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=07S3LI0NTS38 type=del size=824695 2024-05-09T20:17:00.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.308+0000 7fdc882e58c0 1 < suggested update operation key=BS0T1OQOBS67 tag=1TCFNFO1C8CN type=del size=2864253 2024-05-09T20:17:00.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.316+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~VYZ218KA7GNM.0 size=1048576 2024-05-09T20:17:00.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.320+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~VYZ218KA7GNM.1 size=386416 2024-05-09T20:17:00.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.320+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=VYZ218KA7GNM tag=U99TVTNFD55B type=write size=2483568 2024-05-09T20:17:00.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.324+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=VGNDV9AVDB7N type=del 2024-05-09T20:17:00.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.328+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=SWTTF0Q5Z2YH type=del size=1603599 2024-05-09T20:17:00.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.332+0000 7fdc882e58c0 1 < suggested remove operation key=P2DNA4DGI54P tag=7WGLEC2Z8JJ5 type=del 2024-05-09T20:17:00.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.344+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~7G5U2OZ7R2T4.0 size=1048576 2024-05-09T20:17:00.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.348+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~7G5U2OZ7R2T4.1 size=1048576 2024-05-09T20:17:00.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.348+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~7G5U2OZ7R2T4.2 size=113237 2024-05-09T20:17:00.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.348+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=7G5U2OZ7R2T4 tag=BST2N6L2LE3E type=write size=3258965 2024-05-09T20:17:00.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.352+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=3NB7B439L62H type=del 2024-05-09T20:17:00.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.360+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~7L6N0Y4MBS2V.0 size=734016 2024-05-09T20:17:00.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.360+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=7L6N0Y4MBS2V tag=RA84I8M12FBJ type=write size=1782592 2024-05-09T20:17:00.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.364+0000 7fdc882e58c0 1 < canceled operation key=NBAECV9L85UK tag=I3PG9T3XG6W6 type=del size=2500889 2024-05-09T20:17:00.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.368+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=TIO4I0Y9JMZQ type=del size=3046459 2024-05-09T20:17:00.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.368+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=0UOLLGYE1XU3 type=del 2024-05-09T20:17:00.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.372+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=AD1NWXNNL63B type=del size=3680202 2024-05-09T20:17:00.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.380+0000 7fdc882e58c0 1 < suggested update operation key=NYHR6GTPQYZS tag=LHQD98CLPPBK type=del size=2036586 2024-05-09T20:17:00.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.388+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~AXY5CT1XJDST.0 size=426445 2024-05-09T20:17:00.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.388+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=AXY5CT1XJDST tag=XN6RYF6LQWCR type=write size=1475021 2024-05-09T20:17:00.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.388+0000 7fdc882e58c0 1 < canceled part key=_multipart_P2DNA4DGI54P.2~JKCSRCH0DE79.0 size=1048576 2024-05-09T20:17:00.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.392+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~JKCSRCH0DE79.1 size=1048576 2024-05-09T20:17:00.385 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.392+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~JKCSRCH0DE79.2 size=732187 2024-05-09T20:17:00.391 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.400+0000 7fdc882e58c0 1 < canceled multipart upload key=P2DNA4DGI54P upload=JKCSRCH0DE79 tag=6EIQA8N6X58P type=write size=3877915 2024-05-09T20:17:00.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.404+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~EYO9PIXJKJE8.0 size=1048576 2024-05-09T20:17:00.400 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.408+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~EYO9PIXJKJE8.1 size=221197 2024-05-09T20:17:00.400 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.408+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=EYO9PIXJKJE8 tag=QFZYIZUYQTWV type=write size=2318349 2024-05-09T20:17:00.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.412+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~T3ES05YY4XCX.0 size=137628 2024-05-09T20:17:00.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.416+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=T3ES05YY4XCX tag=U9NWCBBDZU25 type=write size=1186204 2024-05-09T20:17:00.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~1RWM94D02DRX.0 size=1048576 2024-05-09T20:17:00.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~1RWM94D02DRX.1 size=282715 2024-05-09T20:17:00.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.424+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=1RWM94D02DRX tag=SOD8CUZQHXIO type=write size=2379867 2024-05-09T20:17:00.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.428+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~IXG9N137NZI2.0 size=150235 2024-05-09T20:17:00.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.436+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=IXG9N137NZI2 tag=T78KCWYU4E2Q type=write size=1198811 2024-05-09T20:17:00.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.440+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~W2F1I3K1MWTY.0 size=1048576 2024-05-09T20:17:00.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.444+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~W2F1I3K1MWTY.1 size=19500 2024-05-09T20:17:00.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.444+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=W2F1I3K1MWTY tag=A6U3GB3U238Y type=write size=2116652 2024-05-09T20:17:00.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.444+0000 7fdc882e58c0 1 < canceled part key=_multipart_5UMJFJRKUPKR.2~JJYYOHD6L7KT.0 size=1048576 2024-05-09T20:17:00.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.448+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~JJYYOHD6L7KT.1 size=208730 2024-05-09T20:17:00.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.452+0000 7fdc882e58c0 1 < canceled multipart upload key=5UMJFJRKUPKR upload=JJYYOHD6L7KT tag=0YBWE201Y6EM type=write size=2305882 2024-05-09T20:17:00.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.456+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=CB2E0OHV2R6N type=del size=3511170 2024-05-09T20:17:00.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.464+0000 7fdc882e58c0 1 < suggested update operation key=G0DX1F257RLD tag=NCTBTOTC70KO type=write size=659996 2024-05-09T20:17:00.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.468+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~2RKP9HXRCB5J.0 size=1048576 2024-05-09T20:17:00.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.472+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~2RKP9HXRCB5J.1 size=1048576 2024-05-09T20:17:00.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.476+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~2RKP9HXRCB5J.2 size=209272 2024-05-09T20:17:00.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.476+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=2RKP9HXRCB5J tag=A1RZFMO9GKL2 type=write size=3355000 2024-05-09T20:17:00.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.480+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=FYT0CJB5FOOT type=del 2024-05-09T20:17:00.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.480+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=BG99IM01YRQT type=del size=2265434 2024-05-09T20:17:00.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.484+0000 7fdc882e58c0 1 < canceled operation key=5UMJFJRKUPKR tag=UYOYMAXY7C6E type=del size=928062 2024-05-09T20:17:00.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.488+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=M8YTNG9TNF2R type=del size=3667270 2024-05-09T20:17:00.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.492+0000 7fdc882e58c0 1 < completed write operation key=5H2GWM9LQELZ tag=2YCBRQLI35L1 type=write size=370315 2024-05-09T20:17:00.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.500+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~AP6AHO7KP0JK.0 size=282711 2024-05-09T20:17:00.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.500+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=AP6AHO7KP0JK tag=P8GBIKK7RNFK type=write size=1331287 2024-05-09T20:17:00.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.504+0000 7fdc882e58c0 1 < suggested update operation key=OGVA2778MJWU tag=07S3LI0NTS38 type=del size=824695 2024-05-09T20:17:00.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.512+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~UI7TD71ITFJ1.0 size=1048576 2024-05-09T20:17:00.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.516+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~UI7TD71ITFJ1.1 size=671839 2024-05-09T20:17:00.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.516+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=UI7TD71ITFJ1 tag=WIYNXAWQ9G3O type=write size=2768991 2024-05-09T20:17:00.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.520+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~VYZ218KA7GNM.0 size=1048576 2024-05-09T20:17:00.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.520+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~VYZ218KA7GNM.1 size=386416 2024-05-09T20:17:00.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.528+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=VYZ218KA7GNM tag=U99TVTNFD55B type=write size=2483568 2024-05-09T20:17:00.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.532+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=RV2G4FBV3PL1 type=del size=845306 2024-05-09T20:17:00.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.536+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=SWTTF0Q5Z2YH type=del 2024-05-09T20:17:00.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.536+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=ZS9XUIV7YJ32 type=del size=2560412 2024-05-09T20:17:00.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.540+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~7G5U2OZ7R2T4.0 size=1048576 2024-05-09T20:17:00.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.544+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~7G5U2OZ7R2T4.1 size=1048576 2024-05-09T20:17:00.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.548+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~7G5U2OZ7R2T4.2 size=113237 2024-05-09T20:17:00.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.552+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=7G5U2OZ7R2T4 tag=BST2N6L2LE3E type=write size=3258965 2024-05-09T20:17:00.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.556+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=AFA0ENCEO7KZ type=del size=2197068 2024-05-09T20:17:00.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.560+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~7L6N0Y4MBS2V.0 size=734016 2024-05-09T20:17:00.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.564+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=7L6N0Y4MBS2V tag=RA84I8M12FBJ type=write size=1782592 2024-05-09T20:17:00.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.568+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=CSQRXXUZTUW5 type=del size=2541101 2024-05-09T20:17:00.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.572+0000 7fdc882e58c0 -1 < failed to complete operation key=OGVA2778MJWU tag=TIO4I0Y9JMZQ type=del size=3046459: (22) Invalid argument 2024-05-09T20:17:00.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.580+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~1X7XNETXPKER.0 size=1048576 2024-05-09T20:17:00.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.580+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~1X7XNETXPKER.1 size=663983 2024-05-09T20:17:00.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.580+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=1X7XNETXPKER tag=SS7U2RUI4X5L type=write size=2761135 2024-05-09T20:17:00.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.584+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=AD1NWXNNL63B type=del 2024-05-09T20:17:00.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.588+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=UMABYPVJJZLD type=del size=883507 2024-05-09T20:17:00.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.592+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~AXY5CT1XJDST.0 size=426445 2024-05-09T20:17:00.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.596+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=AXY5CT1XJDST tag=XN6RYF6LQWCR type=write size=1475021 2024-05-09T20:17:00.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.604+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~0RH288RH5O3H.0 size=589096 2024-05-09T20:17:00.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.604+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=0RH288RH5O3H tag=HVTJ769S4XO9 type=write size=1637672 2024-05-09T20:17:00.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.608+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~EYO9PIXJKJE8.0 size=1048576 2024-05-09T20:17:00.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.612+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~EYO9PIXJKJE8.1 size=221197 2024-05-09T20:17:00.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.616+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=EYO9PIXJKJE8 tag=QFZYIZUYQTWV type=write size=2318349 2024-05-09T20:17:00.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.624+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~6X3NF5FV78WW.0 size=1048576 2024-05-09T20:17:00.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.624+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~6X3NF5FV78WW.1 size=1048576 2024-05-09T20:17:00.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.628+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~6X3NF5FV78WW.2 size=224267 2024-05-09T20:17:00.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.628+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=6X3NF5FV78WW tag=9EAE18TC3DHA type=write size=3369995 2024-05-09T20:17:00.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.632+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~1RWM94D02DRX.0 size=1048576 2024-05-09T20:17:00.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.636+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~1RWM94D02DRX.1 size=282715 2024-05-09T20:17:00.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.640+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=1RWM94D02DRX tag=SOD8CUZQHXIO type=write size=2379867 2024-05-09T20:17:00.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.644+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=HUKK83D1UIS2 type=del size=2542041 2024-05-09T20:17:00.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.648+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~W2F1I3K1MWTY.0 size=1048576 2024-05-09T20:17:00.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.648+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~W2F1I3K1MWTY.1 size=19500 2024-05-09T20:17:00.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.652+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=W2F1I3K1MWTY tag=A6U3GB3U238Y type=write size=2116652 2024-05-09T20:17:00.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.664+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~T7GM9PZPIFAH.0 size=1015845 2024-05-09T20:17:00.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.664+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=T7GM9PZPIFAH tag=O09IB6HC32G0 type=write size=2064421 2024-05-09T20:17:00.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.668+0000 7fdc882e58c0 1 < suggested update operation key=O07UHAIJZF0R tag=CB2E0OHV2R6N type=del size=3511170 2024-05-09T20:17:00.664 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.672+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=H0IEUD1GDA0D type=del size=469516 2024-05-09T20:17:00.668 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.676+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~2RKP9HXRCB5J.0 size=1048576 2024-05-09T20:17:00.668 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.676+0000 7fdc882e58c0 1 < canceled part key=_multipart_NBAECV9L85UK.2~2RKP9HXRCB5J.1 size=1048576 2024-05-09T20:17:00.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.680+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~2RKP9HXRCB5J.2 size=209272 2024-05-09T20:17:00.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.684+0000 7fdc882e58c0 1 < canceled multipart upload key=NBAECV9L85UK upload=2RKP9HXRCB5J tag=A1RZFMO9GKL2 type=write size=3355000 2024-05-09T20:17:00.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.688+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=433AJKEMYZP5 type=del size=1026464 2024-05-09T20:17:00.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.696+0000 7fdc882e58c0 1 < suggested update operation key=EIMPFFK4I6NZ tag=BG99IM01YRQT type=del size=2265434 2024-05-09T20:17:00.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.700+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=F5107NLP6EK9 type=write size=1028731 2024-05-09T20:17:00.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.704+0000 7fdc882e58c0 1 < suggested update operation key=JS90W6PIREMD tag=M8YTNG9TNF2R type=del size=3667270 2024-05-09T20:17:00.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.708+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=JZTIVK946FR6 type=write size=477819 2024-05-09T20:17:00.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.712+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~AP6AHO7KP0JK.0 size=282711 2024-05-09T20:17:00.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.716+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=AP6AHO7KP0JK tag=P8GBIKK7RNFK type=write size=1331287 2024-05-09T20:17:00.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.720+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=R1QGNLRSBDCA type=del size=2602838 2024-05-09T20:17:00.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.724+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~UI7TD71ITFJ1.0 size=1048576 2024-05-09T20:17:00.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.728+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~UI7TD71ITFJ1.1 size=671839 2024-05-09T20:17:00.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.736+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=UI7TD71ITFJ1 tag=WIYNXAWQ9G3O type=write size=2768991 2024-05-09T20:17:00.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.736+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=E1RTZDVRR8FO type=del size=1080967 2024-05-09T20:17:00.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.740+0000 7fdc882e58c0 1 < canceled operation key=SPXHCWZICCWY tag=RV2G4FBV3PL1 type=del size=845306 2024-05-09T20:17:00.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.744+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~224GY37K8XO1.0 size=1048576 2024-05-09T20:17:00.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.748+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~224GY37K8XO1.1 size=1048576 2024-05-09T20:17:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.748+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~224GY37K8XO1.2 size=993272 2024-05-09T20:17:00.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.748+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=224GY37K8XO1 tag=QH3ZHQDBR2QI type=write size=4139000 2024-05-09T20:17:00.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.752+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=ZS9XUIV7YJ32 type=del 2024-05-09T20:17:00.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.760+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~TOD24AOQYYV1.0 size=1048576 2024-05-09T20:17:00.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.760+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~TOD24AOQYYV1.1 size=1048576 2024-05-09T20:17:00.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.764+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~TOD24AOQYYV1.2 size=358769 2024-05-09T20:17:00.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.764+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=TOD24AOQYYV1 tag=0US01OBWO9Z9 type=write size=3504497 2024-05-09T20:17:00.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.764+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=AFA0ENCEO7KZ type=del 2024-05-09T20:17:00.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.768+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=Z52CKX1ZC2B6 type=del size=3317958 2024-05-09T20:17:00.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.772+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=CSQRXXUZTUW5 type=del 2024-05-09T20:17:00.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.776+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~IDJGGII1O97W.0 size=1048576 2024-05-09T20:17:00.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~IDJGGII1O97W.1 size=712435 2024-05-09T20:17:00.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.780+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=IDJGGII1O97W tag=1KI6AIQ8UZGH type=write size=2809587 2024-05-09T20:17:00.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.780+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~1X7XNETXPKER.0 size=1048576 2024-05-09T20:17:00.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.784+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~1X7XNETXPKER.1 size=663983 2024-05-09T20:17:00.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.788+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=1X7XNETXPKER tag=SS7U2RUI4X5L type=write size=2761135 2024-05-09T20:17:00.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.792+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=2Y14YGNQK1U3 type=del size=494130 2024-05-09T20:17:00.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.800+0000 7fdc882e58c0 1 < suggested update operation key=P2DNA4DGI54P tag=UMABYPVJJZLD type=del size=883507 2024-05-09T20:17:00.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.800+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=1YR6IU4FQEBZ type=del size=4089902 2024-05-09T20:17:00.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.804+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~0RH288RH5O3H.0 size=589096 2024-05-09T20:17:00.801 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.808+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=0RH288RH5O3H tag=HVTJ769S4XO9 type=write size=1637672 2024-05-09T20:17:00.806 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.816+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~ATNQADZL2R98.0 size=1048576 2024-05-09T20:17:00.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.816+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~ATNQADZL2R98.1 size=951476 2024-05-09T20:17:00.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.816+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=ATNQADZL2R98 tag=Q5X2Q9W1L0NS type=write size=3048628 2024-05-09T20:17:00.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.820+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~6X3NF5FV78WW.0 size=1048576 2024-05-09T20:17:00.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.820+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~6X3NF5FV78WW.1 size=1048576 2024-05-09T20:17:00.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.824+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~6X3NF5FV78WW.2 size=224267 2024-05-09T20:17:00.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.828+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=6X3NF5FV78WW tag=9EAE18TC3DHA type=write size=3369995 2024-05-09T20:17:00.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.832+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=88JIR045W0ON type=del size=2316474 2024-05-09T20:17:00.830 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.840+0000 7fdc882e58c0 1 < suggested remove operation key=TQ5ZHATLNVPP tag=HUKK83D1UIS2 type=del 2024-05-09T20:17:00.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.844+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~1H3G381UC9J2.0 size=1048576 2024-05-09T20:17:00.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.844+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~1H3G381UC9J2.1 size=1048576 2024-05-09T20:17:00.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.848+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~1H3G381UC9J2.2 size=816658 2024-05-09T20:17:00.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.848+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=1H3G381UC9J2 tag=PJK3GJGVIW8O type=write size=3962386 2024-05-09T20:17:00.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.852+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~T7GM9PZPIFAH.0 size=1015845 2024-05-09T20:17:00.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.856+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=T7GM9PZPIFAH tag=O09IB6HC32G0 type=write size=2064421 2024-05-09T20:17:00.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.860+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=F0AZ2S3QBFE5 type=del size=2371072 2024-05-09T20:17:00.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.864+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=H0IEUD1GDA0D type=del 2024-05-09T20:17:00.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.864+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=FH5JWLV6G3MD type=write size=305533 2024-05-09T20:17:00.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.868+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=433AJKEMYZP5 type=del 2024-05-09T20:17:00.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.872+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=JVQOG7SVPMEG type=del size=1659621 2024-05-09T20:17:00.870 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.880+0000 7fdc882e58c0 1 < suggested remove operation key=TWWOL5D8C1MQ tag=F5107NLP6EK9 type=write 2024-05-09T20:17:00.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.880+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=U4XIDCBDJHMS type=del size=2426160 2024-05-09T20:17:00.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.884+0000 7fdc882e58c0 1 < completed write operation key=EK78CNKUHNKO tag=JZTIVK946FR6 type=write size=477819 2024-05-09T20:17:00.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.888+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=RQPBX55Q9OXC type=del size=3384516 2024-05-09T20:17:00.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.892+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=R1QGNLRSBDCA type=del 2024-05-09T20:17:00.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.896+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~UXBTDEHH5NJ0.0 size=1048576 2024-05-09T20:17:00.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~UXBTDEHH5NJ0.1 size=1048576 2024-05-09T20:17:00.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.904+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~UXBTDEHH5NJ0.2 size=664429 2024-05-09T20:17:00.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.904+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=UXBTDEHH5NJ0 tag=SVA2Z066NN50 type=write size=3810157 2024-05-09T20:17:00.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.908+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=E1RTZDVRR8FO type=del 2024-05-09T20:17:00.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.912+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~U20XYIEHOD2C.0 size=1048576 2024-05-09T20:17:00.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.916+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~U20XYIEHOD2C.1 size=491625 2024-05-09T20:17:00.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.916+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=U20XYIEHOD2C tag=DTKN3QN5MH5F type=write size=2588777 2024-05-09T20:17:00.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.920+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~224GY37K8XO1.0 size=1048576 2024-05-09T20:17:00.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.920+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~224GY37K8XO1.1 size=1048576 2024-05-09T20:17:00.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.924+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~224GY37K8XO1.2 size=993272 2024-05-09T20:17:00.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.928+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=224GY37K8XO1 tag=QH3ZHQDBR2QI type=write size=4139000 2024-05-09T20:17:00.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.932+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=1PVLZY97U3EJ type=write size=406061 2024-05-09T20:17:00.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.936+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~TOD24AOQYYV1.0 size=1048576 2024-05-09T20:17:00.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.940+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~TOD24AOQYYV1.1 size=1048576 2024-05-09T20:17:00.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.940+0000 7fdc882e58c0 1 < canceled part key=_multipart_5H2GWM9LQELZ.2~TOD24AOQYYV1.2 size=358769 2024-05-09T20:17:00.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.944+0000 7fdc882e58c0 1 < canceled multipart upload key=5H2GWM9LQELZ upload=TOD24AOQYYV1 tag=0US01OBWO9Z9 type=write size=3504497 2024-05-09T20:17:00.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.948+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=I1TNET302YZS type=del size=3230828 2024-05-09T20:17:00.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.952+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=Z52CKX1ZC2B6 type=del 2024-05-09T20:17:00.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.960+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=VQ6WY194ZN8T type=del size=225265 2024-05-09T20:17:00.957 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.964+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~IDJGGII1O97W.0 size=1048576 2024-05-09T20:17:00.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.968+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~IDJGGII1O97W.1 size=712435 2024-05-09T20:17:00.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.972+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=IDJGGII1O97W tag=1KI6AIQ8UZGH type=write size=2809587 2024-05-09T20:17:00.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~F100UD6T38EU.0 size=1048576 2024-05-09T20:17:00.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.984+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~F100UD6T38EU.1 size=465973 2024-05-09T20:17:00.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.984+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=F100UD6T38EU tag=4NJNNFDA94ZY type=write size=2563125 2024-05-09T20:17:00.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.988+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=2Y14YGNQK1U3 type=del 2024-05-09T20:17:00.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.992+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=NBHOU33YW6US type=del size=2569569 2024-05-09T20:17:00.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:00.996+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=1YR6IU4FQEBZ type=del 2024-05-09T20:17:00.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.000+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=GXJNLDUTSRHE type=del size=1448741 2024-05-09T20:17:00.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.004+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~ATNQADZL2R98.0 size=1048576 2024-05-09T20:17:00.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.004+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~ATNQADZL2R98.1 size=951476 2024-05-09T20:17:01.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.012+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=ATNQADZL2R98 tag=Q5X2Q9W1L0NS type=write size=3048628 2024-05-09T20:17:01.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.016+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=W2E1BWLGA3WN type=del size=2549280 2024-05-09T20:17:01.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.024+0000 7fdc882e58c0 1 < suggested remove operation key=CNV7NHP27JZ7 tag=88JIR045W0ON type=del 2024-05-09T20:17:01.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.024+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=YNJQE7SSKJ2M type=del size=2793715 2024-05-09T20:17:01.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.028+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~1H3G381UC9J2.0 size=1048576 2024-05-09T20:17:01.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.032+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~1H3G381UC9J2.1 size=1048576 2024-05-09T20:17:01.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.036+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~1H3G381UC9J2.2 size=816658 2024-05-09T20:17:01.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.040+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=1H3G381UC9J2 tag=PJK3GJGVIW8O type=write size=3962386 2024-05-09T20:17:01.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.044+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=E28YKF66KS0J type=write size=380845 2024-05-09T20:17:01.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.052+0000 7fdc882e58c0 1 < suggested update operation key=FNB9B18LIA8L tag=F0AZ2S3QBFE5 type=del size=2371072 2024-05-09T20:17:01.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.056+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=DW7XIY3S5LO5 type=del size=565445 2024-05-09T20:17:01.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.060+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=FH5JWLV6G3MD type=write size=305533 2024-05-09T20:17:01.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.060+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=KKQPG1MXUY6N type=del size=3527283 2024-05-09T20:17:01.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.064+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=JVQOG7SVPMEG type=del 2024-05-09T20:17:01.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.072+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~L0RGFCBRF119.0 size=1048576 2024-05-09T20:17:01.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.072+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~L0RGFCBRF119.1 size=111150 2024-05-09T20:17:01.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.072+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=L0RGFCBRF119 tag=8215NKQZMJ2Y type=write size=2208302 2024-05-09T20:17:01.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.076+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=U4XIDCBDJHMS type=del 2024-05-09T20:17:01.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.080+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=A7O7NIMTWX56 type=del size=2731163 2024-05-09T20:17:01.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.084+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=RQPBX55Q9OXC type=del 2024-05-09T20:17:01.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.088+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=7B7MQ3XTCO05 type=del size=2851200 2024-05-09T20:17:01.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.092+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~UXBTDEHH5NJ0.0 size=1048576 2024-05-09T20:17:01.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.096+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~UXBTDEHH5NJ0.1 size=1048576 2024-05-09T20:17:01.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.100+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~UXBTDEHH5NJ0.2 size=664429 2024-05-09T20:17:01.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.104+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=UXBTDEHH5NJ0 tag=SVA2Z066NN50 type=write size=3810157 2024-05-09T20:17:01.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.112+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~1SPN5OAISJAJ.0 size=1048576 2024-05-09T20:17:01.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.116+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~1SPN5OAISJAJ.1 size=717205 2024-05-09T20:17:01.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.116+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=1SPN5OAISJAJ tag=02OG9E74TR09 type=write size=2814357 2024-05-09T20:17:01.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.120+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~U20XYIEHOD2C.0 size=1048576 2024-05-09T20:17:01.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.120+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~U20XYIEHOD2C.1 size=491625 2024-05-09T20:17:01.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.128+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=U20XYIEHOD2C tag=DTKN3QN5MH5F type=write size=2588777 2024-05-09T20:17:01.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.136+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~Y2KHZ7V915KB.0 size=1048576 2024-05-09T20:17:01.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.136+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~Y2KHZ7V915KB.1 size=1048576 2024-05-09T20:17:01.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.140+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~Y2KHZ7V915KB.2 size=17836 2024-05-09T20:17:01.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.140+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=Y2KHZ7V915KB tag=XSLEE0T2L6SM type=write size=3163564 2024-05-09T20:17:01.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.144+0000 7fdc882e58c0 1 < completed write operation key=8FFZZDURUWM1 tag=1PVLZY97U3EJ type=write size=406061 2024-05-09T20:17:01.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.148+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=X6N9KD691R03 type=del size=621277 2024-05-09T20:17:01.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.152+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=I1TNET302YZS type=del 2024-05-09T20:17:01.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.156+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=T44B951HMRWE type=write size=844744 2024-05-09T20:17:01.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.160+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=VQ6WY194ZN8T type=del 2024-05-09T20:17:01.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.164+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=G0UAM2YMP2OO type=write size=527424 2024-05-09T20:17:01.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.168+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~F100UD6T38EU.0 size=1048576 2024-05-09T20:17:01.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.172+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~F100UD6T38EU.1 size=465973 2024-05-09T20:17:01.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.176+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=F100UD6T38EU tag=4NJNNFDA94ZY type=write size=2563125 2024-05-09T20:17:01.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.180+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=H45HZ87WJIMG type=del size=919954 2024-05-09T20:17:01.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.188+0000 7fdc882e58c0 1 < suggested update operation key=G0DX1F257RLD tag=NBHOU33YW6US type=del size=2569569 2024-05-09T20:17:01.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.192+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=WHRM2BB2GFQI type=del size=2230414 2024-05-09T20:17:01.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.196+0000 7fdc882e58c0 1 < canceled operation key=T6T3IOW9VPA0 tag=GXJNLDUTSRHE type=del size=1448741 2024-05-09T20:17:01.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.200+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=IFLV5KSE8HHF type=del size=1396190 2024-05-09T20:17:01.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.204+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=W2E1BWLGA3WN type=del 2024-05-09T20:17:01.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.204+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=7GJYK2WVDCOW type=del size=3503009 2024-05-09T20:17:01.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.208+0000 7fdc882e58c0 -1 < failed to complete operation key=FNB9B18LIA8L tag=YNJQE7SSKJ2M type=del size=2793715: (22) Invalid argument 2024-05-09T20:17:01.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.216+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~7U27NA1ZQMG0.0 size=1048576 2024-05-09T20:17:01.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.220+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~7U27NA1ZQMG0.1 size=1048576 2024-05-09T20:17:01.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.220+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~7U27NA1ZQMG0.2 size=124483 2024-05-09T20:17:01.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.220+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=7U27NA1ZQMG0 tag=RBJFJMWUICJJ type=write size=3270211 2024-05-09T20:17:01.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.224+0000 7fdc882e58c0 1 < completed write operation key=NNGWUSCETMH4 tag=E28YKF66KS0J type=write size=380845 2024-05-09T20:17:01.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.228+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=IONWJYX2XUQL type=del size=2318657 2024-05-09T20:17:01.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.232+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=DW7XIY3S5LO5 type=del 2024-05-09T20:17:01.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.236+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=NB33G8C170EE type=del size=1608337 2024-05-09T20:17:01.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.240+0000 7fdc882e58c0 1 < suggested update operation key=OGVA2778MJWU tag=KKQPG1MXUY6N type=del size=3527283 2024-05-09T20:17:01.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.248+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~C5QR972G1G9E.0 size=1048576 2024-05-09T20:17:01.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.252+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~C5QR972G1G9E.1 size=1048576 2024-05-09T20:17:01.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.252+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~C5QR972G1G9E.2 size=685572 2024-05-09T20:17:01.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.252+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=C5QR972G1G9E tag=HGRLOKWPT03U type=write size=3831300 2024-05-09T20:17:01.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.256+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~L0RGFCBRF119.0 size=1048576 2024-05-09T20:17:01.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.260+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~L0RGFCBRF119.1 size=111150 2024-05-09T20:17:01.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.264+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=L0RGFCBRF119 tag=8215NKQZMJ2Y type=write size=2208302 2024-05-09T20:17:01.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.272+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~7ZS2IK1ASERR.0 size=1048576 2024-05-09T20:17:01.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.276+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~7ZS2IK1ASERR.1 size=1048576 2024-05-09T20:17:01.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.276+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~7ZS2IK1ASERR.2 size=1025810 2024-05-09T20:17:01.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.276+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=7ZS2IK1ASERR tag=CD8DJ7ST07DD type=write size=4171538 2024-05-09T20:17:01.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.284+0000 7fdc882e58c0 1 < suggested update operation key=T6T3IOW9VPA0 tag=A7O7NIMTWX56 type=del size=2731163 2024-05-09T20:17:01.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.288+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=SN10ZLW39GIF type=del size=4032510 2024-05-09T20:17:01.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.300+0000 7fdc882e58c0 1 < suggested update operation key=EIMPFFK4I6NZ tag=7B7MQ3XTCO05 type=del size=2851200 2024-05-09T20:17:01.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.308+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~V07NRULG04ME.0 size=1048576 2024-05-09T20:17:01.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.308+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~V07NRULG04ME.1 size=1048576 2024-05-09T20:17:01.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.312+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~V07NRULG04ME.2 size=633224 2024-05-09T20:17:01.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.312+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=V07NRULG04ME tag=H0JZIDY1UCDZ type=write size=3778952 2024-05-09T20:17:01.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.312+0000 7fdc882e58c0 1 < canceled part key=_multipart_5H2GWM9LQELZ.2~1SPN5OAISJAJ.0 size=1048576 2024-05-09T20:17:01.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.316+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~1SPN5OAISJAJ.1 size=717205 2024-05-09T20:17:01.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.320+0000 7fdc882e58c0 1 < canceled multipart upload key=5H2GWM9LQELZ upload=1SPN5OAISJAJ tag=02OG9E74TR09 type=write size=2814357 2024-05-09T20:17:01.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.328+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~34SF9EMSB0VB.0 size=1048576 2024-05-09T20:17:01.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.332+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~34SF9EMSB0VB.1 size=1048576 2024-05-09T20:17:01.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.336+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~34SF9EMSB0VB.2 size=1046490 2024-05-09T20:17:01.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.336+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=34SF9EMSB0VB tag=KFHBV4FL4CB1 type=write size=4192218 2024-05-09T20:17:01.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.340+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~Y2KHZ7V915KB.0 size=1048576 2024-05-09T20:17:01.333 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.340+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~Y2KHZ7V915KB.1 size=1048576 2024-05-09T20:17:01.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.344+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~Y2KHZ7V915KB.2 size=17836 2024-05-09T20:17:01.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.352+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=Y2KHZ7V915KB tag=XSLEE0T2L6SM type=write size=3163564 2024-05-09T20:17:01.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.356+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~5FGVFCETEKH9.0 size=1048576 2024-05-09T20:17:01.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.360+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~5FGVFCETEKH9.1 size=1048576 2024-05-09T20:17:01.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.360+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~5FGVFCETEKH9.2 size=153564 2024-05-09T20:17:01.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.360+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=5FGVFCETEKH9 tag=5U6EI7OLW011 type=write size=3299292 2024-05-09T20:17:01.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.364+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=X6N9KD691R03 type=del 2024-05-09T20:17:01.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.372+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~EG7YQ1M5NZEO.0 size=1048576 2024-05-09T20:17:01.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.372+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~EG7YQ1M5NZEO.1 size=1048576 2024-05-09T20:17:01.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.376+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~EG7YQ1M5NZEO.2 size=338612 2024-05-09T20:17:01.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.376+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=EG7YQ1M5NZEO tag=BYOD0TOG72IG type=write size=3484340 2024-05-09T20:17:01.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.380+0000 7fdc882e58c0 1 < completed write operation key=5NEOR8OOFAFZ tag=T44B951HMRWE type=write size=844744 2024-05-09T20:17:01.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.388+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~5MF9S730Q193.0 size=1048576 2024-05-09T20:17:01.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.392+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~5MF9S730Q193.1 size=1048576 2024-05-09T20:17:01.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.396+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~5MF9S730Q193.2 size=745671 2024-05-09T20:17:01.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.396+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=5MF9S730Q193 tag=BAOPO77NQ6OR type=write size=3891399 2024-05-09T20:17:01.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.396+0000 7fdc882e58c0 1 < completed write operation key=NNGWUSCETMH4 tag=G0UAM2YMP2OO type=write size=527424 2024-05-09T20:17:01.392 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.400+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=7VDFE7X30H8M type=del size=1514646 2024-05-09T20:17:01.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.404+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=H45HZ87WJIMG type=del 2024-05-09T20:17:01.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.408+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~L7DJK33LEW3G.0 size=1048576 2024-05-09T20:17:01.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.412+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~L7DJK33LEW3G.1 size=489079 2024-05-09T20:17:01.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.412+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=L7DJK33LEW3G tag=Z7YV1LOV9JWG type=write size=2586231 2024-05-09T20:17:01.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.416+0000 7fdc882e58c0 -1 < failed to complete operation key=OGVA2778MJWU tag=WHRM2BB2GFQI type=del size=2230414: (22) Invalid argument 2024-05-09T20:17:01.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.420+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=RA881JCD9A2K type=del size=2133072 2024-05-09T20:17:01.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.424+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=IFLV5KSE8HHF type=del 2024-05-09T20:17:01.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.428+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~31XJAS6U5Z2A.0 size=1048576 2024-05-09T20:17:01.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.432+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~31XJAS6U5Z2A.1 size=956934 2024-05-09T20:17:01.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.432+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=31XJAS6U5Z2A tag=VNI8IPWWIUXJ type=write size=3054086 2024-05-09T20:17:01.430 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.440+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=7GJYK2WVDCOW type=del size=3503009 2024-05-09T20:17:01.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.444+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=0BLS0N6RAO2Y type=del size=3717823 2024-05-09T20:17:01.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.444+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~7U27NA1ZQMG0.0 size=1048576 2024-05-09T20:17:01.440 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.448+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~7U27NA1ZQMG0.1 size=1048576 2024-05-09T20:17:01.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.452+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~7U27NA1ZQMG0.2 size=124483 2024-05-09T20:17:01.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.456+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=7U27NA1ZQMG0 tag=RBJFJMWUICJJ type=write size=3270211 2024-05-09T20:17:01.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.460+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=89NQGK07CG1C type=write size=895493 2024-05-09T20:17:01.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.464+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=IONWJYX2XUQL type=del 2024-05-09T20:17:01.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.468+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=2GAADTQ8ZNLF type=del size=990811 2024-05-09T20:17:01.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.468+0000 7fdc882e58c0 -1 < failed to complete operation key=EIMPFFK4I6NZ tag=NB33G8C170EE type=del size=1608337: (22) Invalid argument 2024-05-09T20:17:01.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.472+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=T1X2GRYM94T7 type=write size=452706 2024-05-09T20:17:01.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.476+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~C5QR972G1G9E.0 size=1048576 2024-05-09T20:17:01.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.476+0000 7fdc882e58c0 1 < canceled part key=_multipart_VZ7XOAT899GU.2~C5QR972G1G9E.1 size=1048576 2024-05-09T20:17:01.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.476+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~C5QR972G1G9E.2 size=685572 2024-05-09T20:17:01.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.480+0000 7fdc882e58c0 1 < canceled multipart upload key=VZ7XOAT899GU upload=C5QR972G1G9E tag=HGRLOKWPT03U type=write size=3831300 2024-05-09T20:17:01.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~BLWQXCUY5XYP.0 size=1048576 2024-05-09T20:17:01.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~BLWQXCUY5XYP.1 size=1036199 2024-05-09T20:17:01.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.484+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=BLWQXCUY5XYP tag=FF7L9UMR60CS type=write size=3133351 2024-05-09T20:17:01.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.488+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~7ZS2IK1ASERR.0 size=1048576 2024-05-09T20:17:01.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.488+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~7ZS2IK1ASERR.1 size=1048576 2024-05-09T20:17:01.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.488+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~7ZS2IK1ASERR.2 size=1025810 2024-05-09T20:17:01.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.492+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=7ZS2IK1ASERR tag=CD8DJ7ST07DD type=write size=4171538 2024-05-09T20:17:01.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.496+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=66GSBD6FEXTM type=del size=3154699 2024-05-09T20:17:01.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.496+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=SN10ZLW39GIF type=del 2024-05-09T20:17:01.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.500+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=MGZHE8NZNZQF type=del size=2078563 2024-05-09T20:17:01.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.500+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~V07NRULG04ME.0 size=1048576 2024-05-09T20:17:01.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.500+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~V07NRULG04ME.1 size=1048576 2024-05-09T20:17:01.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.504+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~V07NRULG04ME.2 size=633224 2024-05-09T20:17:01.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.508+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=V07NRULG04ME tag=H0JZIDY1UCDZ type=write size=3778952 2024-05-09T20:17:01.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.508+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~DAQ04G4QPDMB.0 size=1048576 2024-05-09T20:17:01.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.512+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~DAQ04G4QPDMB.1 size=1048576 2024-05-09T20:17:01.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.512+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~DAQ04G4QPDMB.2 size=294596 2024-05-09T20:17:01.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.512+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=DAQ04G4QPDMB tag=MZM2N5L1YKSI type=write size=3440324 2024-05-09T20:17:01.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.512+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~34SF9EMSB0VB.0 size=1048576 2024-05-09T20:17:01.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.516+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~34SF9EMSB0VB.1 size=1048576 2024-05-09T20:17:01.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.516+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~34SF9EMSB0VB.2 size=1046490 2024-05-09T20:17:01.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.520+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=34SF9EMSB0VB tag=KFHBV4FL4CB1 type=write size=4192218 2024-05-09T20:17:01.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.524+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~Z41VUU7RZSNX.0 size=1048576 2024-05-09T20:17:01.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.524+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~Z41VUU7RZSNX.1 size=1048576 2024-05-09T20:17:01.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.524+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~Z41VUU7RZSNX.2 size=76229 2024-05-09T20:17:01.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.524+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=Z41VUU7RZSNX tag=FS5RGLOZZD1A type=write size=3221957 2024-05-09T20:17:01.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.528+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~5FGVFCETEKH9.0 size=1048576 2024-05-09T20:17:01.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.528+0000 7fdc882e58c0 1 < canceled part key=_multipart_5UMJFJRKUPKR.2~5FGVFCETEKH9.1 size=1048576 2024-05-09T20:17:01.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.528+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~5FGVFCETEKH9.2 size=153564 2024-05-09T20:17:01.523 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.532+0000 7fdc882e58c0 1 < canceled multipart upload key=5UMJFJRKUPKR upload=5FGVFCETEKH9 tag=5U6EI7OLW011 type=write size=3299292 2024-05-09T20:17:01.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.532+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=FYZA7RAJP3SG type=write size=214108 2024-05-09T20:17:01.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.536+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~EG7YQ1M5NZEO.0 size=1048576 2024-05-09T20:17:01.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.536+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~EG7YQ1M5NZEO.1 size=1048576 2024-05-09T20:17:01.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.536+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~EG7YQ1M5NZEO.2 size=338612 2024-05-09T20:17:01.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.540+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=EG7YQ1M5NZEO tag=BYOD0TOG72IG type=write size=3484340 2024-05-09T20:17:01.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.548+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~B2LX2KDN0GHB.0 size=1048576 2024-05-09T20:17:01.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.548+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~B2LX2KDN0GHB.1 size=1048576 2024-05-09T20:17:01.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.548+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~B2LX2KDN0GHB.2 size=1012489 2024-05-09T20:17:01.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.548+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=B2LX2KDN0GHB tag=UQKQ3O372FP0 type=write size=4158217 2024-05-09T20:17:01.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.552+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~5MF9S730Q193.0 size=1048576 2024-05-09T20:17:01.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.552+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~5MF9S730Q193.1 size=1048576 2024-05-09T20:17:01.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.556+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~5MF9S730Q193.2 size=745671 2024-05-09T20:17:01.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.556+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=5MF9S730Q193 tag=BAOPO77NQ6OR type=write size=3891399 2024-05-09T20:17:01.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.560+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~Z69ZBRDSY2DI.0 size=1048576 2024-05-09T20:17:01.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.564+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~Z69ZBRDSY2DI.1 size=502465 2024-05-09T20:17:01.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.564+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=Z69ZBRDSY2DI tag=TLLLS8ZRFMGH type=write size=2599617 2024-05-09T20:17:01.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.564+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=7VDFE7X30H8M type=del 2024-05-09T20:17:01.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.568+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=J2AHLFTA5GIQ type=write size=404301 2024-05-09T20:17:01.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.568+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~L7DJK33LEW3G.0 size=1048576 2024-05-09T20:17:01.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.572+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~L7DJK33LEW3G.1 size=489079 2024-05-09T20:17:01.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.576+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=L7DJK33LEW3G tag=Z7YV1LOV9JWG type=write size=2586231 2024-05-09T20:17:01.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.576+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=4ZBFK58UOFTF type=del size=4074851 2024-05-09T20:17:01.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.580+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=RA881JCD9A2K type=del 2024-05-09T20:17:01.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.580+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=6IEXS7EUJ96G type=write size=712956 2024-05-09T20:17:01.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.584+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~31XJAS6U5Z2A.0 size=1048576 2024-05-09T20:17:01.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.584+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~31XJAS6U5Z2A.1 size=956934 2024-05-09T20:17:01.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.588+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=31XJAS6U5Z2A tag=VNI8IPWWIUXJ type=write size=3054086 2024-05-09T20:17:01.583 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.592+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=BKVYEN87Z3R2 type=del size=1400709 2024-05-09T20:17:01.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.596+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=0BLS0N6RAO2Y type=del 2024-05-09T20:17:01.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.600+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~W5Y2GYZCX0E2.0 size=677257 2024-05-09T20:17:01.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.600+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=W5Y2GYZCX0E2 tag=LO4W9PN78FBN type=write size=1725833 2024-05-09T20:17:01.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.600+0000 7fdc882e58c0 1 < canceled operation key=TQ5ZHATLNVPP tag=89NQGK07CG1C type=write size=895493 2024-05-09T20:17:01.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.604+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~65K0W20DFHLV.0 size=1048576 2024-05-09T20:17:01.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.608+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~65K0W20DFHLV.1 size=1048576 2024-05-09T20:17:01.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.608+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~65K0W20DFHLV.2 size=134473 2024-05-09T20:17:01.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.608+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=65K0W20DFHLV tag=DPQFXAFDCNPU type=write size=3280201 2024-05-09T20:17:01.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.612+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=2GAADTQ8ZNLF type=del 2024-05-09T20:17:01.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.616+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~BW743WOHWK4E.0 size=942565 2024-05-09T20:17:01.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.616+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=BW743WOHWK4E tag=64O6QLUAJN42 type=write size=1991141 2024-05-09T20:17:01.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.620+0000 7fdc882e58c0 1 < suggested update operation key=CNV7NHP27JZ7 tag=T1X2GRYM94T7 type=write size=452706 2024-05-09T20:17:01.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.620+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=B8Q69STRABO8 type=del size=3762506 2024-05-09T20:17:01.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.624+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~BLWQXCUY5XYP.0 size=1048576 2024-05-09T20:17:01.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.624+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~BLWQXCUY5XYP.1 size=1036199 2024-05-09T20:17:01.621 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.628+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=BLWQXCUY5XYP tag=FF7L9UMR60CS type=write size=3133351 2024-05-09T20:17:01.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.632+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~3WPY0CG22FEM.0 size=1048576 2024-05-09T20:17:01.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.636+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~3WPY0CG22FEM.1 size=1048576 2024-05-09T20:17:01.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.636+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~3WPY0CG22FEM.2 size=866486 2024-05-09T20:17:01.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.636+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=3WPY0CG22FEM tag=XFG8MW1RL4XD type=write size=4012214 2024-05-09T20:17:01.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.640+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=66GSBD6FEXTM type=del 2024-05-09T20:17:01.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~WIKCAYZCG5EZ.0 size=1048576 2024-05-09T20:17:01.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~WIKCAYZCG5EZ.1 size=1048576 2024-05-09T20:17:01.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.648+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~WIKCAYZCG5EZ.2 size=257213 2024-05-09T20:17:01.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.648+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=WIKCAYZCG5EZ tag=BRQFQBDO47QI type=write size=3402941 2024-05-09T20:17:01.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.652+0000 7fdc882e58c0 1 < suggested remove operation key=T5EL7X0P450L tag=MGZHE8NZNZQF type=del 2024-05-09T20:17:01.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.656+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=JYZPKTXO31PB type=del size=1186162 2024-05-09T20:17:01.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.656+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~DAQ04G4QPDMB.0 size=1048576 2024-05-09T20:17:01.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.660+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~DAQ04G4QPDMB.1 size=1048576 2024-05-09T20:17:01.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.660+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~DAQ04G4QPDMB.2 size=294596 2024-05-09T20:17:01.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.664+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=DAQ04G4QPDMB tag=MZM2N5L1YKSI type=write size=3440324 2024-05-09T20:17:01.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.668+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~IBN9QL1ZH3WS.0 size=1048576 2024-05-09T20:17:01.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.672+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~IBN9QL1ZH3WS.1 size=1048576 2024-05-09T20:17:01.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.672+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~IBN9QL1ZH3WS.2 size=1031830 2024-05-09T20:17:01.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.672+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=IBN9QL1ZH3WS tag=ZCA86519PNGD type=write size=4177558 2024-05-09T20:17:01.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.676+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~Z41VUU7RZSNX.0 size=1048576 2024-05-09T20:17:01.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.680+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~Z41VUU7RZSNX.1 size=1048576 2024-05-09T20:17:01.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.680+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~Z41VUU7RZSNX.2 size=76229 2024-05-09T20:17:01.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.688+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=Z41VUU7RZSNX tag=FS5RGLOZZD1A type=write size=3221957 2024-05-09T20:17:01.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.692+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=532MY2SSBGHA type=del size=2208317 2024-05-09T20:17:01.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.692+0000 7fdc882e58c0 1 < completed write operation key=346HV1XQKCOZ tag=FYZA7RAJP3SG type=write size=214108 2024-05-09T20:17:01.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.696+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=EIN6I6ELPDK7 type=del size=1007605 2024-05-09T20:17:01.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.700+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~B2LX2KDN0GHB.0 size=1048576 2024-05-09T20:17:01.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.700+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~B2LX2KDN0GHB.1 size=1048576 2024-05-09T20:17:01.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.704+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~B2LX2KDN0GHB.2 size=1012489 2024-05-09T20:17:01.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.708+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=B2LX2KDN0GHB tag=UQKQ3O372FP0 type=write size=4158217 2024-05-09T20:17:01.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.712+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=WGAHTZFKKWMY type=write size=293022 2024-05-09T20:17:01.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.716+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~Z69ZBRDSY2DI.0 size=1048576 2024-05-09T20:17:01.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.716+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~Z69ZBRDSY2DI.1 size=502465 2024-05-09T20:17:01.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.724+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=Z69ZBRDSY2DI tag=TLLLS8ZRFMGH type=write size=2599617 2024-05-09T20:17:01.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.728+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~OOR5UAIZJY0L.0 size=17381 2024-05-09T20:17:01.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.728+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=OOR5UAIZJY0L tag=DMHGU2FUDNGZ type=write size=1065957 2024-05-09T20:17:01.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.732+0000 7fdc882e58c0 1 < canceled operation key=HPK2T0629B1P tag=J2AHLFTA5GIQ type=write size=404301 2024-05-09T20:17:01.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~1W2U2EIZ5ARZ.0 size=1048576 2024-05-09T20:17:01.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.740+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~1W2U2EIZ5ARZ.1 size=679766 2024-05-09T20:17:01.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.740+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=1W2U2EIZ5ARZ tag=LKJMAXRMDNB6 type=write size=2776918 2024-05-09T20:17:01.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.740+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=4ZBFK58UOFTF type=del 2024-05-09T20:17:01.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.748+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~1QQN1F09YU2B.0 size=335390 2024-05-09T20:17:01.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.748+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=1QQN1F09YU2B tag=G2AMSNL0E65F type=write size=1383966 2024-05-09T20:17:01.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.748+0000 7fdc882e58c0 1 < completed write operation key=BS0T1OQOBS67 tag=6IEXS7EUJ96G type=write size=712956 2024-05-09T20:17:01.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.756+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~H6BSAYY61ONL.0 size=858072 2024-05-09T20:17:01.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.756+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=H6BSAYY61ONL tag=J4AHBGI84J9R type=write size=1906648 2024-05-09T20:17:01.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.760+0000 7fdc882e58c0 1 < suggested remove operation key=O07UHAIJZF0R tag=BKVYEN87Z3R2 type=del 2024-05-09T20:17:01.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.764+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~VV3I1U8XVMLM.0 size=1048576 2024-05-09T20:17:01.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.768+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~VV3I1U8XVMLM.1 size=1048576 2024-05-09T20:17:01.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.768+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~VV3I1U8XVMLM.2 size=27825 2024-05-09T20:17:01.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.768+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=VV3I1U8XVMLM tag=ZEE10WOX0EJQ type=write size=3173553 2024-05-09T20:17:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.776+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~W5Y2GYZCX0E2.0 size=677257 2024-05-09T20:17:01.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.780+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=W5Y2GYZCX0E2 tag=LO4W9PN78FBN type=write size=1725833 2024-05-09T20:17:01.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~XAOM9VDJJ6IE.0 size=375822 2024-05-09T20:17:01.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.784+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=XAOM9VDJJ6IE tag=Q68U1F0GPQ28 type=write size=1424398 2024-05-09T20:17:01.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.788+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~65K0W20DFHLV.0 size=1048576 2024-05-09T20:17:01.782 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.792+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~65K0W20DFHLV.1 size=1048576 2024-05-09T20:17:01.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.792+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~65K0W20DFHLV.2 size=134473 2024-05-09T20:17:01.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.796+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=65K0W20DFHLV tag=DPQFXAFDCNPU type=write size=3280201 2024-05-09T20:17:01.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.800+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=KYG2X00STPQA type=del size=2790156 2024-05-09T20:17:01.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.804+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~BW743WOHWK4E.0 size=942565 2024-05-09T20:17:01.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.808+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=BW743WOHWK4E tag=64O6QLUAJN42 type=write size=1991141 2024-05-09T20:17:01.801 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.808+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=VONBRZPJ9B1Z type=del size=3618646 2024-05-09T20:17:01.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.812+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=B8Q69STRABO8 type=del 2024-05-09T20:17:01.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.816+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~F5L8J5FV0JAD.0 size=1048576 2024-05-09T20:17:01.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.820+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~F5L8J5FV0JAD.1 size=968067 2024-05-09T20:17:01.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.820+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=F5L8J5FV0JAD tag=662K19TPWJI0 type=write size=3065219 2024-05-09T20:17:01.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.824+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~3WPY0CG22FEM.0 size=1048576 2024-05-09T20:17:01.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.824+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~3WPY0CG22FEM.1 size=1048576 2024-05-09T20:17:01.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.828+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~3WPY0CG22FEM.2 size=866486 2024-05-09T20:17:01.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.832+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=3WPY0CG22FEM tag=XFG8MW1RL4XD type=write size=4012214 2024-05-09T20:17:01.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.832+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=Q2MXVR5LRW22 type=del size=1336586 2024-05-09T20:17:01.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.836+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~WIKCAYZCG5EZ.0 size=1048576 2024-05-09T20:17:01.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.836+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~WIKCAYZCG5EZ.1 size=1048576 2024-05-09T20:17:01.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.840+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~WIKCAYZCG5EZ.2 size=257213 2024-05-09T20:17:01.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.844+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=WIKCAYZCG5EZ tag=BRQFQBDO47QI type=write size=3402941 2024-05-09T20:17:01.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.844+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=22HVPXICE855 type=del size=3985405 2024-05-09T20:17:01.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.848+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=JYZPKTXO31PB type=del 2024-05-09T20:17:01.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.852+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~JLSUMD4W7WT2.0 size=1048576 2024-05-09T20:17:01.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.856+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~JLSUMD4W7WT2.1 size=1048576 2024-05-09T20:17:01.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.856+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~JLSUMD4W7WT2.2 size=34006 2024-05-09T20:17:01.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.856+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=JLSUMD4W7WT2 tag=8E3V7971QP4V type=write size=3179734 2024-05-09T20:17:01.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.860+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~IBN9QL1ZH3WS.0 size=1048576 2024-05-09T20:17:01.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.864+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~IBN9QL1ZH3WS.1 size=1048576 2024-05-09T20:17:01.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.864+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~IBN9QL1ZH3WS.2 size=1031830 2024-05-09T20:17:01.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.872+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=IBN9QL1ZH3WS tag=ZCA86519PNGD type=write size=4177558 2024-05-09T20:17:01.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.872+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=1ZQ52ZCQ528O type=del size=3480183 2024-05-09T20:17:01.871 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.880+0000 7fdc882e58c0 1 < suggested remove operation key=BS0T1OQOBS67 tag=532MY2SSBGHA type=del 2024-05-09T20:17:01.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.880+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=5VT0T17DVC39 type=del size=1167130 2024-05-09T20:17:01.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.884+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=EIN6I6ELPDK7 type=del 2024-05-09T20:17:01.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.888+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=9JYRA74TZI0E type=del size=1954490 2024-05-09T20:17:01.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.892+0000 7fdc882e58c0 1 < completed write operation key=NBAECV9L85UK tag=WGAHTZFKKWMY type=write size=293022 2024-05-09T20:17:01.884 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.892+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=2G1PKPNYC2LT type=del size=117207 2024-05-09T20:17:01.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.896+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~OOR5UAIZJY0L.0 size=17381 2024-05-09T20:17:01.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.900+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=OOR5UAIZJY0L tag=DMHGU2FUDNGZ type=write size=1065957 2024-05-09T20:17:01.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.904+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~2RNARFPO0JQR.0 size=1048576 2024-05-09T20:17:01.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.908+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~2RNARFPO0JQR.1 size=1048576 2024-05-09T20:17:01.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.908+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~2RNARFPO0JQR.2 size=186296 2024-05-09T20:17:01.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.908+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=2RNARFPO0JQR tag=74QS3HFKUFSH type=write size=3332024 2024-05-09T20:17:01.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.912+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~1W2U2EIZ5ARZ.0 size=1048576 2024-05-09T20:17:01.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.916+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~1W2U2EIZ5ARZ.1 size=679766 2024-05-09T20:17:01.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.920+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=1W2U2EIZ5ARZ tag=LKJMAXRMDNB6 type=write size=2776918 2024-05-09T20:17:01.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.924+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~U9EKC1133H65.0 size=305277 2024-05-09T20:17:01.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.924+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=U9EKC1133H65 tag=3ZDK69CCE6NS type=write size=1353853 2024-05-09T20:17:01.919 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.928+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~1QQN1F09YU2B.0 size=335390 2024-05-09T20:17:01.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.932+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=1QQN1F09YU2B tag=G2AMSNL0E65F type=write size=1383966 2024-05-09T20:17:01.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~2FHETE2MACAG.0 size=404538 2024-05-09T20:17:01.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.936+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=2FHETE2MACAG tag=P570CT7QBNAU type=write size=1453114 2024-05-09T20:17:01.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.940+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~H6BSAYY61ONL.0 size=858072 2024-05-09T20:17:01.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.944+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=H6BSAYY61ONL tag=J4AHBGI84J9R type=write size=1906648 2024-05-09T20:17:01.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.948+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=ZVOYK04D8H7B type=del size=1954357 2024-05-09T20:17:01.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.948+0000 7fdc882e58c0 1 < canceled part key=_multipart_54R9XPO62J0M.2~VV3I1U8XVMLM.0 size=1048576 2024-05-09T20:17:01.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.948+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~VV3I1U8XVMLM.1 size=1048576 2024-05-09T20:17:01.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.952+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~VV3I1U8XVMLM.2 size=27825 2024-05-09T20:17:01.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.956+0000 7fdc882e58c0 1 < canceled multipart upload key=54R9XPO62J0M upload=VV3I1U8XVMLM tag=ZEE10WOX0EJQ type=write size=3173553 2024-05-09T20:17:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.960+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=CMZYFWTFOT2S type=write size=223355 2024-05-09T20:17:01.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.964+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~XAOM9VDJJ6IE.0 size=375822 2024-05-09T20:17:01.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.972+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=XAOM9VDJJ6IE tag=Q68U1F0GPQ28 type=write size=1424398 2024-05-09T20:17:01.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.976+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~4361ASNEEVMN.0 size=1048576 2024-05-09T20:17:01.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~4361ASNEEVMN.1 size=1048576 2024-05-09T20:17:01.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~4361ASNEEVMN.2 size=865447 2024-05-09T20:17:01.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.980+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=4361ASNEEVMN tag=AFLBTKD7W7WV type=write size=4011175 2024-05-09T20:17:01.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.984+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=KYG2X00STPQA type=del 2024-05-09T20:17:01.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.992+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~0ZVWPZ5CMODD.0 size=736468 2024-05-09T20:17:01.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.992+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=0ZVWPZ5CMODD tag=YNWTDIIY6JKI type=write size=1785044 2024-05-09T20:17:01.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:01.996+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=VONBRZPJ9B1Z type=del 2024-05-09T20:17:01.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.000+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=M0GT5IKRKR06 type=del size=512958 2024-05-09T20:17:01.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.004+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~F5L8J5FV0JAD.0 size=1048576 2024-05-09T20:17:01.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.004+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~F5L8J5FV0JAD.1 size=968067 2024-05-09T20:17:02.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.012+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=F5L8J5FV0JAD tag=662K19TPWJI0 type=write size=3065219 2024-05-09T20:17:02.008 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.016+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~6NLWKE39AGVD.0 size=946271 2024-05-09T20:17:02.008 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.016+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=6NLWKE39AGVD tag=XMU3FXP0CDCB type=write size=1994847 2024-05-09T20:17:02.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.020+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=Q2MXVR5LRW22 type=del 2024-05-09T20:17:02.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.024+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=O70ULBUNAPJC type=del size=3495076 2024-05-09T20:17:02.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.028+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=22HVPXICE855 type=del 2024-05-09T20:17:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.032+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~3V1JAEVDKS9T.0 size=1048576 2024-05-09T20:17:02.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.036+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~3V1JAEVDKS9T.1 size=1048576 2024-05-09T20:17:02.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.040+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~3V1JAEVDKS9T.2 size=242922 2024-05-09T20:17:02.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.040+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=3V1JAEVDKS9T tag=3X7TO159A1RK type=write size=3388650 2024-05-09T20:17:02.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.044+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~JLSUMD4W7WT2.0 size=1048576 2024-05-09T20:17:02.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.048+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~JLSUMD4W7WT2.1 size=1048576 2024-05-09T20:17:02.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.048+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~JLSUMD4W7WT2.2 size=34006 2024-05-09T20:17:02.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.056+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=JLSUMD4W7WT2 tag=8E3V7971QP4V type=write size=3179734 2024-05-09T20:17:02.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.060+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=I9L2AB8KMB8O type=write size=904512 2024-05-09T20:17:02.054 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.064+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=1ZQ52ZCQ528O type=del 2024-05-09T20:17:02.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.064+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=JYGWWTOI6H6X type=del size=956274 2024-05-09T20:17:02.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.068+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=5VT0T17DVC39 type=del 2024-05-09T20:17:02.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.072+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=EU2195SZBDMI type=del size=2589381 2024-05-09T20:17:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.076+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=9JYRA74TZI0E type=del 2024-05-09T20:17:02.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.080+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=ECGJZ763PQQW type=del size=2442412 2024-05-09T20:17:02.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.084+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=2G1PKPNYC2LT type=del 2024-05-09T20:17:02.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.088+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~3RDRYDK8YZFD.0 size=1048576 2024-05-09T20:17:02.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.092+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~3RDRYDK8YZFD.1 size=680632 2024-05-09T20:17:02.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.092+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=3RDRYDK8YZFD tag=ZMK0YYVEE0XH type=write size=2777784 2024-05-09T20:17:02.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.096+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~2RNARFPO0JQR.0 size=1048576 2024-05-09T20:17:02.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.096+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~2RNARFPO0JQR.1 size=1048576 2024-05-09T20:17:02.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.100+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~2RNARFPO0JQR.2 size=186296 2024-05-09T20:17:02.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.104+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=2RNARFPO0JQR tag=74QS3HFKUFSH type=write size=3332024 2024-05-09T20:17:02.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.108+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=HAS0GVBUJ9YX type=del size=3696808 2024-05-09T20:17:02.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.112+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~U9EKC1133H65.0 size=305277 2024-05-09T20:17:02.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.116+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=U9EKC1133H65 tag=3ZDK69CCE6NS type=write size=1353853 2024-05-09T20:17:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.120+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=91HVKCWRYK7G type=del size=4077804 2024-05-09T20:17:02.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.120+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~2FHETE2MACAG.0 size=404538 2024-05-09T20:17:02.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.128+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=2FHETE2MACAG tag=P570CT7QBNAU type=write size=1453114 2024-05-09T20:17:02.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.132+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~AZXXPT63A7YL.0 size=1048576 2024-05-09T20:17:02.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.132+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~AZXXPT63A7YL.1 size=1007098 2024-05-09T20:17:02.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.132+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=AZXXPT63A7YL tag=BFNQYJKAB1V8 type=write size=3104250 2024-05-09T20:17:02.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.136+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=ZVOYK04D8H7B type=del 2024-05-09T20:17:02.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.140+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=XSQZ437YD3OZ type=del size=3401382 2024-05-09T20:17:02.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.144+0000 7fdc882e58c0 1 < canceled operation key=5H2GWM9LQELZ tag=CMZYFWTFOT2S type=write size=223355 2024-05-09T20:17:02.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.148+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=J0WE7DMUKAOF type=del size=292307 2024-05-09T20:17:02.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.148+0000 7fdc882e58c0 1 < canceled part key=_multipart_HPK2T0629B1P.2~4361ASNEEVMN.0 size=1048576 2024-05-09T20:17:02.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.152+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~4361ASNEEVMN.1 size=1048576 2024-05-09T20:17:02.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.152+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~4361ASNEEVMN.2 size=865447 2024-05-09T20:17:02.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.160+0000 7fdc882e58c0 1 < canceled multipart upload key=HPK2T0629B1P upload=4361ASNEEVMN tag=AFLBTKD7W7WV type=write size=4011175 2024-05-09T20:17:02.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.160+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=0KTSYZ659MNK type=del size=2076240 2024-05-09T20:17:02.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.164+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~0ZVWPZ5CMODD.0 size=736468 2024-05-09T20:17:02.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.168+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=0ZVWPZ5CMODD tag=YNWTDIIY6JKI type=write size=1785044 2024-05-09T20:17:02.164 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.172+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=NQ52QXPQ94P3 type=del size=115089 2024-05-09T20:17:02.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.176+0000 7fdc882e58c0 1 < suggested remove operation key=JS90W6PIREMD tag=M0GT5IKRKR06 type=del 2024-05-09T20:17:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.180+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=MKXPRWO67PRZ type=del size=2510109 2024-05-09T20:17:02.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.184+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~6NLWKE39AGVD.0 size=946271 2024-05-09T20:17:02.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.188+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=6NLWKE39AGVD tag=XMU3FXP0CDCB type=write size=1994847 2024-05-09T20:17:02.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.192+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=AZJVWK4RD303 type=del size=629825 2024-05-09T20:17:02.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.196+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=O70ULBUNAPJC type=del 2024-05-09T20:17:02.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.200+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=XN3YDHHLOLXG type=write size=984884 2024-05-09T20:17:02.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.204+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~3V1JAEVDKS9T.0 size=1048576 2024-05-09T20:17:02.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.204+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~3V1JAEVDKS9T.1 size=1048576 2024-05-09T20:17:02.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.208+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~3V1JAEVDKS9T.2 size=242922 2024-05-09T20:17:02.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.212+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=3V1JAEVDKS9T tag=3X7TO159A1RK type=write size=3388650 2024-05-09T20:17:02.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.220+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~FFV9T6IWK5L5.0 size=960676 2024-05-09T20:17:02.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.220+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=FFV9T6IWK5L5 tag=4XVCZHJ7WQ2X type=write size=2009252 2024-05-09T20:17:02.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.224+0000 7fdc882e58c0 1 < completed write operation key=SPXHCWZICCWY tag=I9L2AB8KMB8O type=write size=904512 2024-05-09T20:17:02.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.228+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~DOBN2QMN2OS0.0 size=1048576 2024-05-09T20:17:02.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.232+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~DOBN2QMN2OS0.1 size=185043 2024-05-09T20:17:02.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.232+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=DOBN2QMN2OS0 tag=TOD5VFP1IS2G type=write size=2282195 2024-05-09T20:17:02.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.240+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=JYGWWTOI6H6X type=del size=956274 2024-05-09T20:17:02.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.240+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=9B5OOKAXYNFP type=del size=3875685 2024-05-09T20:17:02.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.244+0000 7fdc882e58c0 -1 < failed to complete operation key=JS90W6PIREMD tag=EU2195SZBDMI type=del size=2589381: (22) Invalid argument 2024-05-09T20:17:02.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.252+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~YLDLRO9F564N.0 size=366351 2024-05-09T20:17:02.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.252+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=YLDLRO9F564N tag=P7DC4X5VEV9Z type=write size=1414927 2024-05-09T20:17:02.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.256+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=ECGJZ763PQQW type=del 2024-05-09T20:17:02.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.260+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=O1RXMBG6GO7X type=del size=518422 2024-05-09T20:17:02.254 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.260+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~3RDRYDK8YZFD.0 size=1048576 2024-05-09T20:17:02.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.264+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~3RDRYDK8YZFD.1 size=680632 2024-05-09T20:17:02.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.268+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=3RDRYDK8YZFD tag=ZMK0YYVEE0XH type=write size=2777784 2024-05-09T20:17:02.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.276+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~2YFTOWMY2SXK.0 size=245074 2024-05-09T20:17:02.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.276+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=2YFTOWMY2SXK tag=3BKQT01MAU81 type=write size=1293650 2024-05-09T20:17:02.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.280+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=HAS0GVBUJ9YX type=del 2024-05-09T20:17:02.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.284+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=RVEGT8E6K3MG type=del size=3555372 2024-05-09T20:17:02.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.284+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=91HVKCWRYK7G type=del 2024-05-09T20:17:02.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.288+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=IYMGX9D7O400 type=del size=3719688 2024-05-09T20:17:02.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.288+0000 7fdc882e58c0 1 < canceled part key=_multipart_346HV1XQKCOZ.2~AZXXPT63A7YL.0 size=1048576 2024-05-09T20:17:02.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.292+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~AZXXPT63A7YL.1 size=1007098 2024-05-09T20:17:02.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.300+0000 7fdc882e58c0 1 < canceled multipart upload key=346HV1XQKCOZ upload=AZXXPT63A7YL tag=BFNQYJKAB1V8 type=write size=3104250 2024-05-09T20:17:02.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.304+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~5TWH9DN43FKX.0 size=136358 2024-05-09T20:17:02.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.304+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=5TWH9DN43FKX tag=IUKAX0O9TF6L type=write size=1184934 2024-05-09T20:17:02.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.308+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=XSQZ437YD3OZ type=del 2024-05-09T20:17:02.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.312+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=NV6A4VCO041T type=del size=1117900 2024-05-09T20:17:02.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.312+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=J0WE7DMUKAOF type=del 2024-05-09T20:17:02.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.316+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=IHCCO0Q0KJ2N type=del size=534386 2024-05-09T20:17:02.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.324+0000 7fdc882e58c0 1 < suggested update operation key=NBAECV9L85UK tag=0KTSYZ659MNK type=del size=2076240 2024-05-09T20:17:02.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.328+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~1UT12X0DE0EZ.0 size=1048576 2024-05-09T20:17:02.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.332+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~1UT12X0DE0EZ.1 size=1048576 2024-05-09T20:17:02.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.336+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~1UT12X0DE0EZ.2 size=240041 2024-05-09T20:17:02.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.336+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=1UT12X0DE0EZ tag=Q5H2CGZY47Z1 type=write size=3385769 2024-05-09T20:17:02.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.340+0000 7fdc882e58c0 -1 < failed to complete operation key=JS90W6PIREMD tag=NQ52QXPQ94P3 type=del size=115089: (22) Invalid argument 2024-05-09T20:17:02.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.344+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=MOVA79169LAC type=write size=281991 2024-05-09T20:17:02.339 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.348+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=MKXPRWO67PRZ type=del 2024-05-09T20:17:02.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.352+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=9G0NA1AHCDX2 type=del size=4044224 2024-05-09T20:17:02.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.352+0000 7fdc882e58c0 -1 < failed to complete operation key=EK78CNKUHNKO tag=AZJVWK4RD303 type=del size=629825: (22) Invalid argument 2024-05-09T20:17:02.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.356+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=MIX9LJSL2W6C type=write size=705201 2024-05-09T20:17:02.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.360+0000 7fdc882e58c0 1 < suggested remove operation key=QM53IENH9CX4 tag=XN3YDHHLOLXG type=write 2024-05-09T20:17:02.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.364+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=GVMWAN590OQS type=write size=651812 2024-05-09T20:17:02.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.368+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~FFV9T6IWK5L5.0 size=960676 2024-05-09T20:17:02.364 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.372+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=FFV9T6IWK5L5 tag=4XVCZHJ7WQ2X type=write size=2009252 2024-05-09T20:17:02.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.376+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=1ZHSZINETPWH type=del size=3542440 2024-05-09T20:17:02.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.380+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~DOBN2QMN2OS0.0 size=1048576 2024-05-09T20:17:02.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.384+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~DOBN2QMN2OS0.1 size=185043 2024-05-09T20:17:02.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.388+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=DOBN2QMN2OS0 tag=TOD5VFP1IS2G type=write size=2282195 2024-05-09T20:17:02.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.392+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=CACS2AIOQ74S type=del size=3289418 2024-05-09T20:17:02.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.396+0000 7fdc882e58c0 1 < suggested remove operation key=O07UHAIJZF0R tag=9B5OOKAXYNFP type=del 2024-05-09T20:17:02.392 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.400+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=VBSMEXO2QRJP type=del size=160256 2024-05-09T20:17:02.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.404+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~YLDLRO9F564N.0 size=366351 2024-05-09T20:17:02.400 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.408+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=YLDLRO9F564N tag=P7DC4X5VEV9Z type=write size=1414927 2024-05-09T20:17:02.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.412+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=2B8NVVVTAJ8H type=del size=2168917 2024-05-09T20:17:02.406 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.416+0000 7fdc882e58c0 1 < canceled operation key=TWWOL5D8C1MQ tag=O1RXMBG6GO7X type=del size=518422 2024-05-09T20:17:02.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.420+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~BJLK2OX3GNNA.0 size=1048576 2024-05-09T20:17:02.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~BJLK2OX3GNNA.1 size=1048576 2024-05-09T20:17:02.416 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~BJLK2OX3GNNA.2 size=474469 2024-05-09T20:17:02.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.424+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=BJLK2OX3GNNA tag=W3N748L9UP0Z type=write size=3620197 2024-05-09T20:17:02.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.428+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~2YFTOWMY2SXK.0 size=245074 2024-05-09T20:17:02.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.432+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=2YFTOWMY2SXK tag=3BKQT01MAU81 type=write size=1293650 2024-05-09T20:17:02.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.436+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=JYB7UJVXQWD9 type=del size=3178977 2024-05-09T20:17:02.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.440+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=RVEGT8E6K3MG type=del 2024-05-09T20:17:02.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.444+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~63AJG53LUHXD.0 size=1048576 2024-05-09T20:17:02.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.448+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~63AJG53LUHXD.1 size=1048576 2024-05-09T20:17:02.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.452+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~63AJG53LUHXD.2 size=177394 2024-05-09T20:17:02.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.452+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=63AJG53LUHXD tag=GEO1BQXIAKTZ type=write size=3323122 2024-05-09T20:17:02.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.452+0000 7fdc882e58c0 1 < canceled operation key=BS0T1OQOBS67 tag=IYMGX9D7O400 type=del size=3719688 2024-05-09T20:17:02.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.460+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~MVWAUY7A3GYH.0 size=1048576 2024-05-09T20:17:02.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.464+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~MVWAUY7A3GYH.1 size=1047919 2024-05-09T20:17:02.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.464+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=MVWAUY7A3GYH tag=07RORWYQ4ZYW type=write size=3145071 2024-05-09T20:17:02.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.468+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~5TWH9DN43FKX.0 size=136358 2024-05-09T20:17:02.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.472+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=5TWH9DN43FKX tag=IUKAX0O9TF6L type=write size=1184934 2024-05-09T20:17:02.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.476+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~RV8RSZZTPBB5.0 size=1048576 2024-05-09T20:17:02.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.480+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~RV8RSZZTPBB5.1 size=1048576 2024-05-09T20:17:02.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~RV8RSZZTPBB5.2 size=243959 2024-05-09T20:17:02.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.484+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=RV8RSZZTPBB5 tag=WSETDDL7G564 type=write size=3389687 2024-05-09T20:17:02.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.488+0000 7fdc882e58c0 1 < suggested remove operation key=EIMPFFK4I6NZ tag=NV6A4VCO041T type=del 2024-05-09T20:17:02.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.492+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=TXNNFPA90C67 type=del size=1076667 2024-05-09T20:17:02.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.496+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=IHCCO0Q0KJ2N type=del 2024-05-09T20:17:02.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.500+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=W99OUSNGSQ6F type=del size=2802764 2024-05-09T20:17:02.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.500+0000 7fdc882e58c0 1 < canceled part key=_multipart_T5EL7X0P450L.2~1UT12X0DE0EZ.0 size=1048576 2024-05-09T20:17:02.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.504+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~1UT12X0DE0EZ.1 size=1048576 2024-05-09T20:17:02.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.504+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~1UT12X0DE0EZ.2 size=240041 2024-05-09T20:17:02.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.512+0000 7fdc882e58c0 1 < canceled multipart upload key=T5EL7X0P450L upload=1UT12X0DE0EZ tag=Q5H2CGZY47Z1 type=write size=3385769 2024-05-09T20:17:02.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.516+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=M17I5J3YMQ31 type=write size=10122 2024-05-09T20:17:02.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.516+0000 7fdc882e58c0 1 < completed write operation key=BS0T1OQOBS67 tag=MOVA79169LAC type=write size=281991 2024-05-09T20:17:02.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.520+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=F9YKFVG8DSZ3 type=write size=890374 2024-05-09T20:17:02.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.524+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=9G0NA1AHCDX2 type=del 2024-05-09T20:17:02.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.528+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=EXRRCCBYT3N8 type=del size=3360241 2024-05-09T20:17:02.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.532+0000 7fdc882e58c0 1 < completed write operation key=WC5YH0OU51WX tag=MIX9LJSL2W6C type=write size=705201 2024-05-09T20:17:02.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.540+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~1D7QC3717XA1.0 size=116497 2024-05-09T20:17:02.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.540+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=1D7QC3717XA1 tag=524USSR5K7BI type=write size=1165073 2024-05-09T20:17:02.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.540+0000 7fdc882e58c0 1 < completed write operation key=J2Q86HKRMAKE tag=GVMWAN590OQS type=write size=651812 2024-05-09T20:17:02.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.548+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~V2E95N4FSO0X.0 size=1048576 2024-05-09T20:17:02.543 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.552+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~V2E95N4FSO0X.1 size=1048576 2024-05-09T20:17:02.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.552+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~V2E95N4FSO0X.2 size=593846 2024-05-09T20:17:02.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.552+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=V2E95N4FSO0X tag=6S4B9SKMBBY6 type=write size=3739574 2024-05-09T20:17:02.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.556+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=1ZHSZINETPWH type=del 2024-05-09T20:17:02.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.564+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~1N1MVDV0O288.0 size=1048576 2024-05-09T20:17:02.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.568+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~1N1MVDV0O288.1 size=133861 2024-05-09T20:17:02.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.568+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=1N1MVDV0O288 tag=DKIU1GJNWDDS type=write size=2231013 2024-05-09T20:17:02.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.572+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=CACS2AIOQ74S type=del 2024-05-09T20:17:02.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.576+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=O8STA6TZZ45O type=del size=3740141 2024-05-09T20:17:02.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.576+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=VBSMEXO2QRJP type=del 2024-05-09T20:17:02.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.580+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=D5II2173IM2H type=del size=2881159 2024-05-09T20:17:02.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.584+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=2B8NVVVTAJ8H type=del 2024-05-09T20:17:02.583 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.592+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~64TA7FA276Z8.0 size=332042 2024-05-09T20:17:02.583 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.592+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=64TA7FA276Z8 tag=S5GJT0O8YZKV type=write size=1380618 2024-05-09T20:17:02.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.596+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~BJLK2OX3GNNA.0 size=1048576 2024-05-09T20:17:02.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.596+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~BJLK2OX3GNNA.1 size=1048576 2024-05-09T20:17:02.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.600+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~BJLK2OX3GNNA.2 size=474469 2024-05-09T20:17:02.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.604+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=BJLK2OX3GNNA tag=W3N748L9UP0Z type=write size=3620197 2024-05-09T20:17:02.603 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.612+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~YJ1418Z056IU.0 size=1048576 2024-05-09T20:17:02.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.616+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~YJ1418Z056IU.1 size=898581 2024-05-09T20:17:02.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.616+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=YJ1418Z056IU tag=GXGY79TPERJF type=write size=2995733 2024-05-09T20:17:02.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.620+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=JYB7UJVXQWD9 type=del 2024-05-09T20:17:02.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.620+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=M8R425RVL4BU type=del size=3829198 2024-05-09T20:17:02.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.624+0000 7fdc882e58c0 1 < canceled part key=_multipart_EK78CNKUHNKO.2~63AJG53LUHXD.0 size=1048576 2024-05-09T20:17:02.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.624+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~63AJG53LUHXD.1 size=1048576 2024-05-09T20:17:02.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.628+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~63AJG53LUHXD.2 size=177394 2024-05-09T20:17:02.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.636+0000 7fdc882e58c0 1 < canceled multipart upload key=EK78CNKUHNKO upload=63AJG53LUHXD tag=GEO1BQXIAKTZ type=write size=3323122 2024-05-09T20:17:02.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.640+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~DS74OD4KBJTC.0 size=1048576 2024-05-09T20:17:02.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~DS74OD4KBJTC.1 size=190568 2024-05-09T20:17:02.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.644+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=DS74OD4KBJTC tag=4WH0LVQA6A45 type=write size=2287720 2024-05-09T20:17:02.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.648+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~MVWAUY7A3GYH.0 size=1048576 2024-05-09T20:17:02.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.648+0000 7fdc882e58c0 1 < canceled part key=_multipart_HPK2T0629B1P.2~MVWAUY7A3GYH.1 size=1047919 2024-05-09T20:17:02.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.656+0000 7fdc882e58c0 1 < canceled multipart upload key=HPK2T0629B1P upload=MVWAUY7A3GYH tag=07RORWYQ4ZYW type=write size=3145071 2024-05-09T20:17:02.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.660+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=EH1Z7JMLJ58U type=del size=1268630 2024-05-09T20:17:02.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.664+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~RV8RSZZTPBB5.0 size=1048576 2024-05-09T20:17:02.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.668+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~RV8RSZZTPBB5.1 size=1048576 2024-05-09T20:17:02.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.668+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~RV8RSZZTPBB5.2 size=243959 2024-05-09T20:17:02.668 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.676+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=RV8RSZZTPBB5 tag=WSETDDL7G564 type=write size=3389687 2024-05-09T20:17:02.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.680+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=2IGP7L7LP615 type=write size=918857 2024-05-09T20:17:02.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.684+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=TXNNFPA90C67 type=del 2024-05-09T20:17:02.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.692+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~S4FL6C367V3Z.0 size=1048576 2024-05-09T20:17:02.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.692+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~S4FL6C367V3Z.1 size=540013 2024-05-09T20:17:02.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.692+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=S4FL6C367V3Z tag=U6TDTBU2L5W3 type=write size=2637165 2024-05-09T20:17:02.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.696+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=W99OUSNGSQ6F type=del 2024-05-09T20:17:02.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.700+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=E3SHDZ2T7WD7 type=del size=937323 2024-05-09T20:17:02.695 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.704+0000 7fdc882e58c0 1 < completed write operation key=EK78CNKUHNKO tag=M17I5J3YMQ31 type=write size=10122 2024-05-09T20:17:02.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.708+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~KAVXCWSE6RFV.0 size=1048576 2024-05-09T20:17:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~KAVXCWSE6RFV.1 size=599319 2024-05-09T20:17:02.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.712+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=KAVXCWSE6RFV tag=13OHIK3MY7LL type=write size=2696471 2024-05-09T20:17:02.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.716+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=F9YKFVG8DSZ3 type=write size=890374 2024-05-09T20:17:02.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.720+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=QQ95ZX52HKA5 type=write size=55742 2024-05-09T20:17:02.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.724+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=EXRRCCBYT3N8 type=del 2024-05-09T20:17:02.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.728+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=W9I5TT71CSQN type=del size=3300979 2024-05-09T20:17:02.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.732+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~1D7QC3717XA1.0 size=116497 2024-05-09T20:17:02.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.736+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=1D7QC3717XA1 tag=524USSR5K7BI type=write size=1165073 2024-05-09T20:17:02.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.744+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~LHI9GV3PI30P.0 size=1048576 2024-05-09T20:17:02.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.744+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~LHI9GV3PI30P.1 size=1048576 2024-05-09T20:17:02.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.748+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~LHI9GV3PI30P.2 size=275629 2024-05-09T20:17:02.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.748+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=LHI9GV3PI30P tag=UBIQAES1454D type=write size=3421357 2024-05-09T20:17:02.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.752+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~V2E95N4FSO0X.0 size=1048576 2024-05-09T20:17:02.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.752+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~V2E95N4FSO0X.1 size=1048576 2024-05-09T20:17:02.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.756+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~V2E95N4FSO0X.2 size=593846 2024-05-09T20:17:02.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.760+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=V2E95N4FSO0X tag=6S4B9SKMBBY6 type=write size=3739574 2024-05-09T20:17:02.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.764+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=L3WB3S9UOVF8 type=write size=478343 2024-05-09T20:17:02.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.764+0000 7fdc882e58c0 1 < canceled part key=_multipart_J2Q86HKRMAKE.2~1N1MVDV0O288.0 size=1048576 2024-05-09T20:17:02.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.768+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~1N1MVDV0O288.1 size=133861 2024-05-09T20:17:02.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.772+0000 7fdc882e58c0 1 < canceled multipart upload key=J2Q86HKRMAKE upload=1N1MVDV0O288 tag=DKIU1GJNWDDS type=write size=2231013 2024-05-09T20:17:02.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~JCR9Y6LI4B40.0 size=1048576 2024-05-09T20:17:02.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~JCR9Y6LI4B40.1 size=1048576 2024-05-09T20:17:02.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.784+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~JCR9Y6LI4B40.2 size=961531 2024-05-09T20:17:02.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.784+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=JCR9Y6LI4B40 tag=YJTTX5GKKBRA type=write size=4107259 2024-05-09T20:17:02.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.788+0000 7fdc882e58c0 1 < canceled operation key=QM53IENH9CX4 tag=O8STA6TZZ45O type=del size=3740141 2024-05-09T20:17:02.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.792+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~7CA877DDMS99.0 size=667980 2024-05-09T20:17:02.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.792+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=7CA877DDMS99 tag=LBHVK4MKEO3R type=write size=1716556 2024-05-09T20:17:02.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.796+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=D5II2173IM2H type=del 2024-05-09T20:17:02.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.804+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~PORCV3MSMT38.0 size=1048576 2024-05-09T20:17:02.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.804+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~PORCV3MSMT38.1 size=1048576 2024-05-09T20:17:02.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.808+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~PORCV3MSMT38.2 size=974177 2024-05-09T20:17:02.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.808+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=PORCV3MSMT38 tag=7B38YO95GSFB type=write size=4119905 2024-05-09T20:17:02.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.812+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~64TA7FA276Z8.0 size=332042 2024-05-09T20:17:02.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.816+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=64TA7FA276Z8 tag=S5GJT0O8YZKV type=write size=1380618 2024-05-09T20:17:02.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.824+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~GTZG4NFQU6E1.0 size=1048576 2024-05-09T20:17:02.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.828+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~GTZG4NFQU6E1.1 size=1048576 2024-05-09T20:17:02.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.828+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~GTZG4NFQU6E1.2 size=983317 2024-05-09T20:17:02.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.828+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=GTZG4NFQU6E1 tag=IPFUAC3RYDDB type=write size=4129045 2024-05-09T20:17:02.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.832+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~YJ1418Z056IU.0 size=1048576 2024-05-09T20:17:02.828 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.836+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~YJ1418Z056IU.1 size=898581 2024-05-09T20:17:02.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.844+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=YJ1418Z056IU tag=GXGY79TPERJF type=write size=2995733 2024-05-09T20:17:02.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.844+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=76TLDA7VWAAZ type=del size=3612837 2024-05-09T20:17:02.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.848+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=M8R425RVL4BU type=del 2024-05-09T20:17:02.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.852+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=D3NTY65WM9UB type=del size=1679878 2024-05-09T20:17:02.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.856+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~DS74OD4KBJTC.0 size=1048576 2024-05-09T20:17:02.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.860+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~DS74OD4KBJTC.1 size=190568 2024-05-09T20:17:02.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.864+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=DS74OD4KBJTC tag=4WH0LVQA6A45 type=write size=2287720 2024-05-09T20:17:02.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.868+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=501QS3L5H0W5 type=del size=2256939 2024-05-09T20:17:02.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.876+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=EH1Z7JMLJ58U type=del size=1268630 2024-05-09T20:17:02.872 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.880+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~CCLVD032H8KL.0 size=1048576 2024-05-09T20:17:02.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.884+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~CCLVD032H8KL.1 size=68985 2024-05-09T20:17:02.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.884+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=CCLVD032H8KL tag=GSM21M6Y5XCK type=write size=2166137 2024-05-09T20:17:02.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.888+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=2IGP7L7LP615 type=write size=918857 2024-05-09T20:17:02.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.888+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=92M7PD4B7F2C type=del size=3178517 2024-05-09T20:17:02.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.892+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~S4FL6C367V3Z.0 size=1048576 2024-05-09T20:17:02.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.896+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~S4FL6C367V3Z.1 size=540013 2024-05-09T20:17:02.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.904+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=S4FL6C367V3Z tag=U6TDTBU2L5W3 type=write size=2637165 2024-05-09T20:17:02.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.904+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=2VXDEKV2B6K1 type=write size=2318 2024-05-09T20:17:02.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.912+0000 7fdc882e58c0 1 < suggested remove operation key=TQ5ZHATLNVPP tag=E3SHDZ2T7WD7 type=del 2024-05-09T20:17:02.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.916+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=DS8VDWD10IIF type=write size=431473 2024-05-09T20:17:02.911 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.920+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~KAVXCWSE6RFV.0 size=1048576 2024-05-09T20:17:02.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.924+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~KAVXCWSE6RFV.1 size=599319 2024-05-09T20:17:02.919 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.928+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=KAVXCWSE6RFV tag=13OHIK3MY7LL type=write size=2696471 2024-05-09T20:17:02.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.932+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~2DR8X6GUZXTQ.0 size=896300 2024-05-09T20:17:02.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.932+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=2DR8X6GUZXTQ tag=0IJN2OJ59SWC type=write size=1944876 2024-05-09T20:17:02.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.936+0000 7fdc882e58c0 1 < canceled operation key=HPK2T0629B1P tag=QQ95ZX52HKA5 type=write size=55742 2024-05-09T20:17:02.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.940+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=658HASKHBR0W type=del size=3054394 2024-05-09T20:17:02.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.944+0000 7fdc882e58c0 -1 < failed to complete operation key=EK78CNKUHNKO tag=W9I5TT71CSQN type=del size=3300979: (22) Invalid argument 2024-05-09T20:17:02.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.944+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=AYXS4N60CJNO type=del size=4116743 2024-05-09T20:17:02.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.948+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~LHI9GV3PI30P.0 size=1048576 2024-05-09T20:17:02.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.952+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~LHI9GV3PI30P.1 size=1048576 2024-05-09T20:17:02.946 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.956+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~LHI9GV3PI30P.2 size=275629 2024-05-09T20:17:02.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.960+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=LHI9GV3PI30P tag=UBIQAES1454D type=write size=3421357 2024-05-09T20:17:02.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.968+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=0ANQZ0HGAJSS type=del size=1000626 2024-05-09T20:17:02.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.972+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=L3WB3S9UOVF8 type=write size=478343 2024-05-09T20:17:02.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.976+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=EDAM0OTHSD4R type=del size=1431305 2024-05-09T20:17:02.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.980+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~JCR9Y6LI4B40.0 size=1048576 2024-05-09T20:17:02.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.984+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~JCR9Y6LI4B40.1 size=1048576 2024-05-09T20:17:02.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.988+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~JCR9Y6LI4B40.2 size=961531 2024-05-09T20:17:02.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:02.992+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=JCR9Y6LI4B40 tag=YJTTX5GKKBRA type=write size=4107259 2024-05-09T20:17:02.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.000+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~AR5IO51FH5MX.0 size=1048576 2024-05-09T20:17:02.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.000+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~AR5IO51FH5MX.1 size=1048576 2024-05-09T20:17:02.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.004+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~AR5IO51FH5MX.2 size=1392 2024-05-09T20:17:02.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.004+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=AR5IO51FH5MX tag=4F8AMGCYESJ6 type=write size=3147120 2024-05-09T20:17:03.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.008+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~7CA877DDMS99.0 size=667980 2024-05-09T20:17:03.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.016+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=7CA877DDMS99 tag=LBHVK4MKEO3R type=write size=1716556 2024-05-09T20:17:03.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.020+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~RQKCNW4TNDHL.0 size=1048576 2024-05-09T20:17:03.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.024+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~RQKCNW4TNDHL.1 size=49706 2024-05-09T20:17:03.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.024+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=RQKCNW4TNDHL tag=VLS49GWCKNMW type=write size=2146858 2024-05-09T20:17:03.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.024+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~PORCV3MSMT38.0 size=1048576 2024-05-09T20:17:03.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.028+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~PORCV3MSMT38.1 size=1048576 2024-05-09T20:17:03.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.032+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~PORCV3MSMT38.2 size=974177 2024-05-09T20:17:03.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.036+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=PORCV3MSMT38 tag=7B38YO95GSFB type=write size=4119905 2024-05-09T20:17:03.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.040+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=DKEJ13CQOXKI type=del size=3928373 2024-05-09T20:17:03.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.044+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~GTZG4NFQU6E1.0 size=1048576 2024-05-09T20:17:03.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.044+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~GTZG4NFQU6E1.1 size=1048576 2024-05-09T20:17:03.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.048+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~GTZG4NFQU6E1.2 size=983317 2024-05-09T20:17:03.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.052+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=GTZG4NFQU6E1 tag=IPFUAC3RYDDB type=write size=4129045 2024-05-09T20:17:03.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.060+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~ARSTL8HNUNV6.0 size=1048576 2024-05-09T20:17:03.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.060+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~ARSTL8HNUNV6.1 size=1048576 2024-05-09T20:17:03.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.064+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~ARSTL8HNUNV6.2 size=130017 2024-05-09T20:17:03.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.064+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=ARSTL8HNUNV6 tag=DR26HHI5XC90 type=write size=3275745 2024-05-09T20:17:03.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.068+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=76TLDA7VWAAZ type=del 2024-05-09T20:17:03.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.072+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=P8JPRD10WDFK type=write size=270864 2024-05-09T20:17:03.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.076+0000 7fdc882e58c0 1 < suggested remove operation key=T6T3IOW9VPA0 tag=D3NTY65WM9UB type=del 2024-05-09T20:17:03.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.084+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~C5281324YVVH.0 size=1048576 2024-05-09T20:17:03.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.084+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~C5281324YVVH.1 size=1048576 2024-05-09T20:17:03.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.088+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~C5281324YVVH.2 size=1928 2024-05-09T20:17:03.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.088+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=C5281324YVVH tag=DYMZF73977AO type=write size=3147656 2024-05-09T20:17:03.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.092+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=501QS3L5H0W5 type=del 2024-05-09T20:17:03.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.096+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=FLR994JZ6CC5 type=del size=3220432 2024-05-09T20:17:03.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.100+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~CCLVD032H8KL.0 size=1048576 2024-05-09T20:17:03.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.100+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~CCLVD032H8KL.1 size=68985 2024-05-09T20:17:03.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.104+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=CCLVD032H8KL tag=GSM21M6Y5XCK type=write size=2166137 2024-05-09T20:17:03.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.108+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=I59J11M4HGVI type=del size=3584685 2024-05-09T20:17:03.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.116+0000 7fdc882e58c0 1 < suggested update operation key=QM53IENH9CX4 tag=92M7PD4B7F2C type=del size=3178517 2024-05-09T20:17:03.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.120+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=BJP7B9G9H113 type=del size=2788962 2024-05-09T20:17:03.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.124+0000 7fdc882e58c0 1 < completed write operation key=NYHR6GTPQYZS tag=2VXDEKV2B6K1 type=write size=2318 2024-05-09T20:17:03.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.128+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=5P4ZUD9Y8G9N type=del size=3957608 2024-05-09T20:17:03.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.128+0000 7fdc882e58c0 -1 < failed to complete operation key=QM53IENH9CX4 tag=DS8VDWD10IIF type=write size=431473: (22) Invalid argument 2024-05-09T20:17:03.124 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.132+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=YQB33247MS6P type=write size=425485 2024-05-09T20:17:03.127 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.136+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~2DR8X6GUZXTQ.0 size=896300 2024-05-09T20:17:03.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.140+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=2DR8X6GUZXTQ tag=0IJN2OJ59SWC type=write size=1944876 2024-05-09T20:17:03.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.140+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=8XMEL40M1N2Z type=del size=1231000 2024-05-09T20:17:03.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.148+0000 7fdc882e58c0 1 < suggested update operation key=CNV7NHP27JZ7 tag=658HASKHBR0W type=del size=3054394 2024-05-09T20:17:03.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.152+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=WC0QDH8USXLK type=del size=1335541 2024-05-09T20:17:03.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.152+0000 7fdc882e58c0 -1 < no bi entry to suggest for operation key=T6T3IOW9VPA0 tag=AYXS4N60CJNO type=del size=4116743: (2) No such file or directory 2024-05-09T20:17:03.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.156+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=JF7YZIFM9XHG type=write size=941191 2024-05-09T20:17:03.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.160+0000 7fdc882e58c0 1 < canceled operation key=T5EL7X0P450L tag=0ANQZ0HGAJSS type=del size=1000626 2024-05-09T20:17:03.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.160+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=N4IHMTVYDCND type=del size=3613342 2024-05-09T20:17:03.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.164+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=EDAM0OTHSD4R type=del 2024-05-09T20:17:03.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.168+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=ODA1OOT17T8J type=del size=749947 2024-05-09T20:17:03.164 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.172+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~AR5IO51FH5MX.0 size=1048576 2024-05-09T20:17:03.164 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.172+0000 7fdc882e58c0 1 < canceled part key=_multipart_EK78CNKUHNKO.2~AR5IO51FH5MX.1 size=1048576 2024-05-09T20:17:03.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.176+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~AR5IO51FH5MX.2 size=1392 2024-05-09T20:17:03.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.180+0000 7fdc882e58c0 1 < canceled multipart upload key=EK78CNKUHNKO upload=AR5IO51FH5MX tag=4F8AMGCYESJ6 type=write size=3147120 2024-05-09T20:17:03.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.188+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~WFZBBV39I3EC.0 size=1048576 2024-05-09T20:17:03.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.188+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~WFZBBV39I3EC.1 size=1048576 2024-05-09T20:17:03.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.192+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~WFZBBV39I3EC.2 size=435433 2024-05-09T20:17:03.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.192+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=WFZBBV39I3EC tag=5J9CVRIU3AZR type=write size=3581161 2024-05-09T20:17:03.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.192+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~RQKCNW4TNDHL.0 size=1048576 2024-05-09T20:17:03.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.196+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~RQKCNW4TNDHL.1 size=49706 2024-05-09T20:17:03.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.200+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=RQKCNW4TNDHL tag=VLS49GWCKNMW type=write size=2146858 2024-05-09T20:17:03.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.204+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=QFHH5L70TLTC type=write size=301480 2024-05-09T20:17:03.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.208+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=DKEJ13CQOXKI type=del 2024-05-09T20:17:03.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.212+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=MZZ0ELK0JD7F type=del size=921737 2024-05-09T20:17:03.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.216+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~ARSTL8HNUNV6.0 size=1048576 2024-05-09T20:17:03.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.216+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~ARSTL8HNUNV6.1 size=1048576 2024-05-09T20:17:03.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.220+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~ARSTL8HNUNV6.2 size=130017 2024-05-09T20:17:03.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.224+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=ARSTL8HNUNV6 tag=DR26HHI5XC90 type=write size=3275745 2024-05-09T20:17:03.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.228+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=XPJCHAZS4E4R type=del size=1036299 2024-05-09T20:17:03.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.232+0000 7fdc882e58c0 1 < completed write operation key=OGVA2778MJWU tag=P8JPRD10WDFK type=write size=270864 2024-05-09T20:17:03.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.236+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=8LGOXP12RYA4 type=del size=195260 2024-05-09T20:17:03.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.240+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~C5281324YVVH.0 size=1048576 2024-05-09T20:17:03.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.240+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~C5281324YVVH.1 size=1048576 2024-05-09T20:17:03.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.244+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~C5281324YVVH.2 size=1928 2024-05-09T20:17:03.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.248+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=C5281324YVVH tag=DYMZF73977AO type=write size=3147656 2024-05-09T20:17:03.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.252+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=2E0YU7AH4SLM type=del size=214476 2024-05-09T20:17:03.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.256+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=FLR994JZ6CC5 type=del 2024-05-09T20:17:03.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.260+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~HG5LQ7VMP1C4.0 size=1048576 2024-05-09T20:17:03.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.264+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~HG5LQ7VMP1C4.1 size=922344 2024-05-09T20:17:03.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.264+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=HG5LQ7VMP1C4 tag=CVSG2TN9Z8AH type=write size=3019496 2024-05-09T20:17:03.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.268+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=I59J11M4HGVI type=del 2024-05-09T20:17:03.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.268+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=1WZ01QWN43II type=del size=3520289 2024-05-09T20:17:03.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.276+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=BJP7B9G9H113 type=del 2024-05-09T20:17:03.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.280+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=OTGL861AQE27 type=del size=2870605 2024-05-09T20:17:03.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.280+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=5P4ZUD9Y8G9N type=del 2024-05-09T20:17:03.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.288+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~817XUZWZCF75.0 size=408559 2024-05-09T20:17:03.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.288+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=817XUZWZCF75 tag=K03H5FROTU73 type=write size=1457135 2024-05-09T20:17:03.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.292+0000 7fdc882e58c0 1 < completed write operation key=JS90W6PIREMD tag=YQB33247MS6P type=write size=425485 2024-05-09T20:17:03.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.292+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=1H6NZI57Q3AW type=del size=1190151 2024-05-09T20:17:03.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.296+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=8XMEL40M1N2Z type=del 2024-05-09T20:17:03.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.300+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=UQBXILAIFFPT type=del size=1166983 2024-05-09T20:17:03.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.304+0000 7fdc882e58c0 1 < canceled operation key=SNPNT5JFJOUS tag=WC0QDH8USXLK type=del size=1335541 2024-05-09T20:17:03.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.308+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=Z8BHEYYO1F2X type=del size=492864 2024-05-09T20:17:03.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.312+0000 7fdc882e58c0 1 < canceled operation key=TWWOL5D8C1MQ tag=JF7YZIFM9XHG type=write size=941191 2024-05-09T20:17:03.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.316+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~99NGW6F4QSK5.0 size=1048576 2024-05-09T20:17:03.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.320+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~99NGW6F4QSK5.1 size=81663 2024-05-09T20:17:03.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.320+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=99NGW6F4QSK5 tag=TLHYVYQCRH1A type=write size=2178815 2024-05-09T20:17:03.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.324+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=N4IHMTVYDCND type=del 2024-05-09T20:17:03.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.328+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=BSQBY67ADJTX type=del size=519698 2024-05-09T20:17:03.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.332+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=ODA1OOT17T8J type=del 2024-05-09T20:17:03.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.340+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~ZRG88GFKGV5E.0 size=56029 2024-05-09T20:17:03.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.340+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=ZRG88GFKGV5E tag=B5TWTHID1L82 type=write size=1104605 2024-05-09T20:17:03.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.344+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~WFZBBV39I3EC.0 size=1048576 2024-05-09T20:17:03.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.344+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~WFZBBV39I3EC.1 size=1048576 2024-05-09T20:17:03.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.348+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~WFZBBV39I3EC.2 size=435433 2024-05-09T20:17:03.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.352+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=WFZBBV39I3EC tag=5J9CVRIU3AZR type=write size=3581161 2024-05-09T20:17:03.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.360+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~EKQ8O4N3H4U8.0 size=1048576 2024-05-09T20:17:03.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.360+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~EKQ8O4N3H4U8.1 size=235675 2024-05-09T20:17:03.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.360+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=EKQ8O4N3H4U8 tag=7HCBNOD13NV4 type=write size=2332827 2024-05-09T20:17:03.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.364+0000 7fdc882e58c0 1 < completed write operation key=JS90W6PIREMD tag=QFHH5L70TLTC type=write size=301480 2024-05-09T20:17:03.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.368+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=934FH8N0WFZZ type=del size=3518787 2024-05-09T20:17:03.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.372+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=MZZ0ELK0JD7F type=del 2024-05-09T20:17:03.366 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.376+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=JIN2ORIZBVQH type=del size=1422426 2024-05-09T20:17:03.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.376+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=XPJCHAZS4E4R type=del 2024-05-09T20:17:03.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.384+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~T4JFKNC30R2I.0 size=1048576 2024-05-09T20:17:03.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.384+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~T4JFKNC30R2I.1 size=1048576 2024-05-09T20:17:03.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.388+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~T4JFKNC30R2I.2 size=773922 2024-05-09T20:17:03.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.388+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=T4JFKNC30R2I tag=U6K5OP2TTJ43 type=write size=3919650 2024-05-09T20:17:03.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.392+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=8LGOXP12RYA4 type=del 2024-05-09T20:17:03.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.392+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=9T0PH90LZ008 type=del size=526918 2024-05-09T20:17:03.391 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.400+0000 7fdc882e58c0 1 < suggested remove operation key=FNB9B18LIA8L tag=2E0YU7AH4SLM type=del 2024-05-09T20:17:03.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.404+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~RS7TSLY8YZQT.0 size=1048576 2024-05-09T20:17:03.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.408+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~RS7TSLY8YZQT.1 size=16738 2024-05-09T20:17:03.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.408+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=RS7TSLY8YZQT tag=C4MTUM4T12PP type=write size=2113890 2024-05-09T20:17:03.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.408+0000 7fdc882e58c0 1 < canceled part key=_multipart_P2DNA4DGI54P.2~HG5LQ7VMP1C4.0 size=1048576 2024-05-09T20:17:03.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.412+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~HG5LQ7VMP1C4.1 size=922344 2024-05-09T20:17:03.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.416+0000 7fdc882e58c0 1 < canceled multipart upload key=P2DNA4DGI54P upload=HG5LQ7VMP1C4 tag=CVSG2TN9Z8AH type=write size=3019496 2024-05-09T20:17:03.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~5AH72H2EP30O.0 size=542862 2024-05-09T20:17:03.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.424+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=5AH72H2EP30O tag=9ZDFYI9UWO8H type=write size=1591438 2024-05-09T20:17:03.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.428+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=1WZ01QWN43II type=del 2024-05-09T20:17:03.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.432+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~Q6Y6K0OK14A1.0 size=458337 2024-05-09T20:17:03.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.432+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=Q6Y6K0OK14A1 tag=RO9KEM6NQPJM type=write size=1506913 2024-05-09T20:17:03.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.436+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=OTGL861AQE27 type=del 2024-05-09T20:17:03.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.440+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~4848UX2GIUX0.0 size=1048576 2024-05-09T20:17:03.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.444+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~4848UX2GIUX0.1 size=600504 2024-05-09T20:17:03.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.444+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=4848UX2GIUX0 tag=BCY0YJBTOUTF type=write size=2697656 2024-05-09T20:17:03.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.448+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~817XUZWZCF75.0 size=408559 2024-05-09T20:17:03.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.452+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=817XUZWZCF75 tag=K03H5FROTU73 type=write size=1457135 2024-05-09T20:17:03.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.456+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~HWO2GK954AF2.0 size=872498 2024-05-09T20:17:03.450 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.456+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=HWO2GK954AF2 tag=F016133R3VS2 type=write size=1921074 2024-05-09T20:17:03.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.460+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=1H6NZI57Q3AW type=del 2024-05-09T20:17:03.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.464+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=5J2NVE5VR5HH type=del size=1388850 2024-05-09T20:17:03.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.468+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=UQBXILAIFFPT type=del 2024-05-09T20:17:03.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.472+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=URURNFQ019AA type=del size=2980072 2024-05-09T20:17:03.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.476+0000 7fdc882e58c0 1 < suggested update operation key=346HV1XQKCOZ tag=Z8BHEYYO1F2X type=del size=492864 2024-05-09T20:17:03.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.480+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~8M78MFPE45P8.0 size=1048576 2024-05-09T20:17:03.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~8M78MFPE45P8.1 size=1048576 2024-05-09T20:17:03.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.484+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~8M78MFPE45P8.2 size=317513 2024-05-09T20:17:03.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.484+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=8M78MFPE45P8 tag=Y49DF1NVMN2P type=write size=3463241 2024-05-09T20:17:03.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.488+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~99NGW6F4QSK5.0 size=1048576 2024-05-09T20:17:03.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.488+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~99NGW6F4QSK5.1 size=81663 2024-05-09T20:17:03.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.496+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=99NGW6F4QSK5 tag=TLHYVYQCRH1A type=write size=2178815 2024-05-09T20:17:03.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.500+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~91QT5DIFILWO.0 size=394964 2024-05-09T20:17:03.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.500+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=91QT5DIFILWO tag=T6XGB6D517EF type=write size=1443540 2024-05-09T20:17:03.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.504+0000 7fdc882e58c0 1 < suggested update operation key=VZ7XOAT899GU tag=BSQBY67ADJTX type=del size=519698 2024-05-09T20:17:03.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.508+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~G1D4O3M1NEL4.0 size=1048576 2024-05-09T20:17:03.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.512+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~G1D4O3M1NEL4.1 size=1048576 2024-05-09T20:17:03.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.516+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~G1D4O3M1NEL4.2 size=630197 2024-05-09T20:17:03.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.516+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=G1D4O3M1NEL4 tag=8P5N43VKL3OJ type=write size=3775925 2024-05-09T20:17:03.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.516+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~ZRG88GFKGV5E.0 size=56029 2024-05-09T20:17:03.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.520+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=ZRG88GFKGV5E tag=B5TWTHID1L82 type=write size=1104605 2024-05-09T20:17:03.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.524+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=XAKE443Y23W7 type=write size=646593 2024-05-09T20:17:03.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.528+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~EKQ8O4N3H4U8.0 size=1048576 2024-05-09T20:17:03.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.528+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~EKQ8O4N3H4U8.1 size=235675 2024-05-09T20:17:03.525 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.532+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=EKQ8O4N3H4U8 tag=7HCBNOD13NV4 type=write size=2332827 2024-05-09T20:17:03.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.536+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=1F6EXDNEA9OZ type=write size=582984 2024-05-09T20:17:03.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.540+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=934FH8N0WFZZ type=del 2024-05-09T20:17:03.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.540+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=KTRYMB23S862 type=write size=161021 2024-05-09T20:17:03.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.544+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=JIN2ORIZBVQH type=del 2024-05-09T20:17:03.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.548+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~5EM2CIALACGD.0 size=890934 2024-05-09T20:17:03.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.548+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=5EM2CIALACGD tag=LBQUL5PRQIQE type=write size=1939510 2024-05-09T20:17:03.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.552+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~T4JFKNC30R2I.0 size=1048576 2024-05-09T20:17:03.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.552+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~T4JFKNC30R2I.1 size=1048576 2024-05-09T20:17:03.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.556+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~T4JFKNC30R2I.2 size=773922 2024-05-09T20:17:03.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.564+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=T4JFKNC30R2I tag=U6K5OP2TTJ43 type=write size=3919650 2024-05-09T20:17:03.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.564+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=EVJFNLYFSVB2 type=write size=613172 2024-05-09T20:17:03.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.572+0000 7fdc882e58c0 1 < suggested remove operation key=SPXHCWZICCWY tag=9T0PH90LZ008 type=del 2024-05-09T20:17:03.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.572+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=MQN1SSZBHI9G type=del size=3523301 2024-05-09T20:17:03.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.576+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~RS7TSLY8YZQT.0 size=1048576 2024-05-09T20:17:03.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.576+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~RS7TSLY8YZQT.1 size=16738 2024-05-09T20:17:03.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.580+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=RS7TSLY8YZQT tag=C4MTUM4T12PP type=write size=2113890 2024-05-09T20:17:03.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~JHHFL1Q8TP32.0 size=1048576 2024-05-09T20:17:03.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.588+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~JHHFL1Q8TP32.1 size=1048576 2024-05-09T20:17:03.582 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.592+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~JHHFL1Q8TP32.2 size=717716 2024-05-09T20:17:03.582 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.592+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=JHHFL1Q8TP32 tag=2EAUUKN2Z6VW type=write size=3863444 2024-05-09T20:17:03.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.592+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~5AH72H2EP30O.0 size=542862 2024-05-09T20:17:03.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.596+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=5AH72H2EP30O tag=9ZDFYI9UWO8H type=write size=1591438 2024-05-09T20:17:03.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.600+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=53SPO5XU99D3 type=write size=32501 2024-05-09T20:17:03.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.604+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~Q6Y6K0OK14A1.0 size=458337 2024-05-09T20:17:03.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.608+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=Q6Y6K0OK14A1 tag=RO9KEM6NQPJM type=write size=1506913 2024-05-09T20:17:03.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.608+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=COQUNC61525T type=write size=378776 2024-05-09T20:17:03.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.612+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~4848UX2GIUX0.0 size=1048576 2024-05-09T20:17:03.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.616+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~4848UX2GIUX0.1 size=600504 2024-05-09T20:17:03.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.620+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=4848UX2GIUX0 tag=BCY0YJBTOUTF type=write size=2697656 2024-05-09T20:17:03.616 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.624+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=9ILME8TZ9KZ6 type=del size=35884 2024-05-09T20:17:03.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.628+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~HWO2GK954AF2.0 size=872498 2024-05-09T20:17:03.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.632+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=HWO2GK954AF2 tag=F016133R3VS2 type=write size=1921074 2024-05-09T20:17:03.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.636+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~YPJURVNTL0YN.0 size=507359 2024-05-09T20:17:03.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.636+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=YPJURVNTL0YN tag=RPTZ8Q6GTEJ8 type=write size=1555935 2024-05-09T20:17:03.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.640+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=5J2NVE5VR5HH type=del 2024-05-09T20:17:03.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~DWK3LKPD9JPD.0 size=1048576 2024-05-09T20:17:03.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.648+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~DWK3LKPD9JPD.1 size=35299 2024-05-09T20:17:03.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.648+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=DWK3LKPD9JPD tag=L22NC20NOJ1S type=write size=2132451 2024-05-09T20:17:03.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.652+0000 7fdc882e58c0 1 < suggested remove operation key=EIMPFFK4I6NZ tag=URURNFQ019AA type=del 2024-05-09T20:17:03.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.656+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=RMPSJNVNI7RY type=del size=4020552 2024-05-09T20:17:03.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.656+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~8M78MFPE45P8.0 size=1048576 2024-05-09T20:17:03.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.656+0000 7fdc882e58c0 1 < canceled part key=_multipart_WC5YH0OU51WX.2~8M78MFPE45P8.1 size=1048576 2024-05-09T20:17:03.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.660+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~8M78MFPE45P8.2 size=317513 2024-05-09T20:17:03.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.664+0000 7fdc882e58c0 1 < canceled multipart upload key=WC5YH0OU51WX upload=8M78MFPE45P8 tag=Y49DF1NVMN2P type=write size=3463241 2024-05-09T20:17:03.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.668+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=P16JSCO9BWBG type=del size=1003223 2024-05-09T20:17:03.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.668+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~91QT5DIFILWO.0 size=394964 2024-05-09T20:17:03.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.676+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=91QT5DIFILWO tag=T6XGB6D517EF type=write size=1443540 2024-05-09T20:17:03.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.680+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~266OIMUZYLQ2.0 size=1048576 2024-05-09T20:17:03.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.684+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~266OIMUZYLQ2.1 size=1048576 2024-05-09T20:17:03.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.684+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~266OIMUZYLQ2.2 size=861234 2024-05-09T20:17:03.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.684+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=266OIMUZYLQ2 tag=1TX2LW1B4G24 type=write size=4006962 2024-05-09T20:17:03.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.688+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~G1D4O3M1NEL4.0 size=1048576 2024-05-09T20:17:03.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.692+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~G1D4O3M1NEL4.1 size=1048576 2024-05-09T20:17:03.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.692+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~G1D4O3M1NEL4.2 size=630197 2024-05-09T20:17:03.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.700+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=G1D4O3M1NEL4 tag=8P5N43VKL3OJ type=write size=3775925 2024-05-09T20:17:03.695 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.704+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=IFB5BWHV3JIR type=del size=4156108 2024-05-09T20:17:03.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.708+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=XAKE443Y23W7 type=write size=646593 2024-05-09T20:17:03.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.712+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~JBYZZ5PCYEWG.0 size=1048576 2024-05-09T20:17:03.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.716+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~JBYZZ5PCYEWG.1 size=625757 2024-05-09T20:17:03.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.716+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=JBYZZ5PCYEWG tag=J00JAZTYL6FN type=write size=2722909 2024-05-09T20:17:03.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.716+0000 7fdc882e58c0 1 < completed write operation key=T6T3IOW9VPA0 tag=1F6EXDNEA9OZ type=write size=582984 2024-05-09T20:17:03.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.720+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=YCHWEOYKE3CF type=del size=2502463 2024-05-09T20:17:03.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.724+0000 7fdc882e58c0 1 < canceled operation key=G0DX1F257RLD tag=KTRYMB23S862 type=write size=161021 2024-05-09T20:17:03.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.728+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=VK65CWKNUZ32 type=write size=922160 2024-05-09T20:17:03.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.732+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~5EM2CIALACGD.0 size=890934 2024-05-09T20:17:03.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.740+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=5EM2CIALACGD tag=LBQUL5PRQIQE type=write size=1939510 2024-05-09T20:17:03.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.744+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~82X3341AYUYH.0 size=1048576 2024-05-09T20:17:03.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.744+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~82X3341AYUYH.1 size=1048576 2024-05-09T20:17:03.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.748+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~82X3341AYUYH.2 size=632665 2024-05-09T20:17:03.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.748+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=82X3341AYUYH tag=T14YVC569VWI type=write size=3778393 2024-05-09T20:17:03.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.752+0000 7fdc882e58c0 1 < completed write operation key=BS0T1OQOBS67 tag=EVJFNLYFSVB2 type=write size=613172 2024-05-09T20:17:03.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.752+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=IA1RDNI6MTN9 type=del size=2471342 2024-05-09T20:17:03.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.756+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=MQN1SSZBHI9G type=del 2024-05-09T20:17:03.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.760+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=8QF8RYCSS0MW type=del size=1355648 2024-05-09T20:17:03.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.764+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~JHHFL1Q8TP32.0 size=1048576 2024-05-09T20:17:03.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.764+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~JHHFL1Q8TP32.1 size=1048576 2024-05-09T20:17:03.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.768+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~JHHFL1Q8TP32.2 size=717716 2024-05-09T20:17:03.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.772+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=JHHFL1Q8TP32 tag=2EAUUKN2Z6VW type=write size=3863444 2024-05-09T20:17:03.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.780+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~FL5P6NS9N7IT.0 size=34693 2024-05-09T20:17:03.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.780+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=FL5P6NS9N7IT tag=KNZOMEWYQOQK type=write size=1083269 2024-05-09T20:17:03.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.784+0000 7fdc882e58c0 1 < suggested remove operation key=SPXHCWZICCWY tag=53SPO5XU99D3 type=write 2024-05-09T20:17:03.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.788+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=F99B6QKKCWPT type=del size=3823357 2024-05-09T20:17:03.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.792+0000 7fdc882e58c0 1 < completed write operation key=NBAECV9L85UK tag=COQUNC61525T type=write size=378776 2024-05-09T20:17:03.786 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.792+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=2ZAM8MM0O2BY type=del size=1478467 2024-05-09T20:17:03.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.796+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=9ILME8TZ9KZ6 type=del 2024-05-09T20:17:03.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.804+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~JSO0FK0R5JLA.0 size=1048576 2024-05-09T20:17:03.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.804+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~JSO0FK0R5JLA.1 size=1048576 2024-05-09T20:17:03.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.808+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~JSO0FK0R5JLA.2 size=142563 2024-05-09T20:17:03.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.808+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=JSO0FK0R5JLA tag=0L7AXK52ZD2W type=write size=3288291 2024-05-09T20:17:03.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.812+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~YPJURVNTL0YN.0 size=507359 2024-05-09T20:17:03.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.816+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=YPJURVNTL0YN tag=RPTZ8Q6GTEJ8 type=write size=1555935 2024-05-09T20:17:03.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.820+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=94UDH3PQB35C type=del size=1355666 2024-05-09T20:17:03.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.820+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~DWK3LKPD9JPD.0 size=1048576 2024-05-09T20:17:03.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.824+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~DWK3LKPD9JPD.1 size=35299 2024-05-09T20:17:03.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.832+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=DWK3LKPD9JPD tag=L22NC20NOJ1S type=write size=2132451 2024-05-09T20:17:03.830 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.836+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~9E1QFFN9CGM5.0 size=772122 2024-05-09T20:17:03.830 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.836+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=9E1QFFN9CGM5 tag=0F8U8CP7OWK1 type=write size=1820698 2024-05-09T20:17:03.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.840+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=RMPSJNVNI7RY type=del 2024-05-09T20:17:03.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.844+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=SW633UT9KLNW type=del size=367832 2024-05-09T20:17:03.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.844+0000 7fdc882e58c0 1 < canceled operation key=SNPNT5JFJOUS tag=P16JSCO9BWBG type=del size=1003223 2024-05-09T20:17:03.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.848+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=CSCTV5A0N55F type=write size=911287 2024-05-09T20:17:03.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.852+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~266OIMUZYLQ2.0 size=1048576 2024-05-09T20:17:03.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.852+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~266OIMUZYLQ2.1 size=1048576 2024-05-09T20:17:03.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.856+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~266OIMUZYLQ2.2 size=861234 2024-05-09T20:17:03.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.860+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=266OIMUZYLQ2 tag=1TX2LW1B4G24 type=write size=4006962 2024-05-09T20:17:03.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.864+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=3UA56884DAW7 type=del size=363251 2024-05-09T20:17:03.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.868+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=IFB5BWHV3JIR type=del 2024-05-09T20:17:03.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.868+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=DEXXXH1J7M0K type=del size=2186908 2024-05-09T20:17:03.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.872+0000 7fdc882e58c0 1 < canceled part key=_multipart_SNPNT5JFJOUS.2~JBYZZ5PCYEWG.0 size=1048576 2024-05-09T20:17:03.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.872+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~JBYZZ5PCYEWG.1 size=625757 2024-05-09T20:17:03.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.876+0000 7fdc882e58c0 1 < canceled multipart upload key=SNPNT5JFJOUS upload=JBYZZ5PCYEWG tag=J00JAZTYL6FN type=write size=2722909 2024-05-09T20:17:03.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.884+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~CZYDSBR64VA0.0 size=1048576 2024-05-09T20:17:03.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.884+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~CZYDSBR64VA0.1 size=1048576 2024-05-09T20:17:03.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.888+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~CZYDSBR64VA0.2 size=199290 2024-05-09T20:17:03.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.888+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=CZYDSBR64VA0 tag=6QI9BOFL3LJ6 type=write size=3345018 2024-05-09T20:17:03.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.892+0000 7fdc882e58c0 -1 < failed to complete operation key=SPXHCWZICCWY tag=YCHWEOYKE3CF type=del size=2502463: (22) Invalid argument 2024-05-09T20:17:03.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.896+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~1D90BX47DNQP.0 size=1048576 2024-05-09T20:17:03.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.896+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~1D90BX47DNQP.1 size=1048576 2024-05-09T20:17:03.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.900+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~1D90BX47DNQP.2 size=678582 2024-05-09T20:17:03.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.900+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=1D90BX47DNQP tag=04M7MCCZGD4Y type=write size=3824310 2024-05-09T20:17:03.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.904+0000 7fdc882e58c0 1 < completed write operation key=IZPPFLWVPRKO tag=VK65CWKNUZ32 type=write size=922160 2024-05-09T20:17:03.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.904+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=H9Y99HTBL5UI type=del size=1752864 2024-05-09T20:17:03.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.908+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~82X3341AYUYH.0 size=1048576 2024-05-09T20:17:03.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.912+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~82X3341AYUYH.1 size=1048576 2024-05-09T20:17:03.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.912+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~82X3341AYUYH.2 size=632665 2024-05-09T20:17:03.909 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.916+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=82X3341AYUYH tag=T14YVC569VWI type=write size=3778393 2024-05-09T20:17:03.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.920+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=EJK6QPDNQNCD type=del size=2766556 2024-05-09T20:17:03.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.924+0000 7fdc882e58c0 1 < suggested remove operation key=EK78CNKUHNKO tag=IA1RDNI6MTN9 type=del 2024-05-09T20:17:03.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.928+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=84HKWUPYDTCJ type=del size=32360 2024-05-09T20:17:03.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.932+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=8QF8RYCSS0MW type=del 2024-05-09T20:17:03.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.936+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=30RHFCCWP1S5 type=del size=1301276 2024-05-09T20:17:03.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.940+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~FL5P6NS9N7IT.0 size=34693 2024-05-09T20:17:03.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.944+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=FL5P6NS9N7IT tag=KNZOMEWYQOQK type=write size=1083269 2024-05-09T20:17:03.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.948+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~099PX5UH3K79.0 size=1048576 2024-05-09T20:17:03.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.952+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~099PX5UH3K79.1 size=319497 2024-05-09T20:17:03.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.952+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=099PX5UH3K79 tag=D2I0OQQH7ZA3 type=write size=2416649 2024-05-09T20:17:03.945 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.952+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=F99B6QKKCWPT type=del 2024-05-09T20:17:03.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.956+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~XHI5F2Q0SCQR.0 size=1048576 2024-05-09T20:17:03.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.960+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~XHI5F2Q0SCQR.1 size=770139 2024-05-09T20:17:03.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.960+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=XHI5F2Q0SCQR tag=AGTR7Z3JPDWQ type=write size=2867291 2024-05-09T20:17:03.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.964+0000 7fdc882e58c0 1 < canceled operation key=54R9XPO62J0M tag=2ZAM8MM0O2BY type=del size=1478467 2024-05-09T20:17:03.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.964+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=70YQCI6G1PSD type=del size=2254758 2024-05-09T20:17:03.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.968+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~JSO0FK0R5JLA.0 size=1048576 2024-05-09T20:17:03.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.968+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~JSO0FK0R5JLA.1 size=1048576 2024-05-09T20:17:03.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.972+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~JSO0FK0R5JLA.2 size=142563 2024-05-09T20:17:03.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.976+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=JSO0FK0R5JLA tag=0L7AXK52ZD2W type=write size=3288291 2024-05-09T20:17:03.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.980+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~33N9Q153OFLM.0 size=1048576 2024-05-09T20:17:03.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.984+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~33N9Q153OFLM.1 size=269084 2024-05-09T20:17:03.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.984+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=33N9Q153OFLM tag=B37R4BZTXN5S type=write size=2366236 2024-05-09T20:17:03.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.988+0000 7fdc882e58c0 1 < suggested remove operation key=AJANS5ADJO5B tag=94UDH3PQB35C type=del 2024-05-09T20:17:03.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.992+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=HB6OZFWU3E31 type=del size=1369224 2024-05-09T20:17:03.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:03.996+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~9E1QFFN9CGM5.0 size=772122 2024-05-09T20:17:03.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.000+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=9E1QFFN9CGM5 tag=0F8U8CP7OWK1 type=write size=1820698 2024-05-09T20:17:03.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.004+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~J0TX0NGS7WF7.0 size=1048576 2024-05-09T20:17:03.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.008+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~J0TX0NGS7WF7.1 size=1048576 2024-05-09T20:17:04.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.008+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~J0TX0NGS7WF7.2 size=529195 2024-05-09T20:17:04.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.008+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=J0TX0NGS7WF7 tag=34I5LZOVOYOZ type=write size=3674923 2024-05-09T20:17:04.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.012+0000 7fdc882e58c0 1 < canceled operation key=J2Q86HKRMAKE tag=SW633UT9KLNW type=del size=367832 2024-05-09T20:17:04.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.016+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=TSXHO9R9YRKQ type=del size=1851002 2024-05-09T20:17:04.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.020+0000 7fdc882e58c0 1 < completed write operation key=NBAECV9L85UK tag=CSCTV5A0N55F type=write size=911287 2024-05-09T20:17:04.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.024+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~VJO7NVDH6KXM.0 size=1048576 2024-05-09T20:17:04.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.024+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~VJO7NVDH6KXM.1 size=707653 2024-05-09T20:17:04.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.024+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=VJO7NVDH6KXM tag=HJHJE5UL7DH9 type=write size=2804805 2024-05-09T20:17:04.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.032+0000 7fdc882e58c0 1 < suggested update operation key=5H2GWM9LQELZ tag=3UA56884DAW7 type=del size=363251 2024-05-09T20:17:04.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.036+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=QILFWYTNGP90 type=del size=1370615 2024-05-09T20:17:04.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.036+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=DEXXXH1J7M0K type=del 2024-05-09T20:17:04.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.040+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=E9V7G5712IW4 type=del size=2642229 2024-05-09T20:17:04.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.040+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~CZYDSBR64VA0.0 size=1048576 2024-05-09T20:17:04.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.044+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~CZYDSBR64VA0.1 size=1048576 2024-05-09T20:17:04.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.044+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~CZYDSBR64VA0.2 size=199290 2024-05-09T20:17:04.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.052+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=CZYDSBR64VA0 tag=6QI9BOFL3LJ6 type=write size=3345018 2024-05-09T20:17:04.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.056+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~5L7C585NA8XZ.0 size=1048576 2024-05-09T20:17:04.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.056+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~5L7C585NA8XZ.1 size=1048576 2024-05-09T20:17:04.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.060+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~5L7C585NA8XZ.2 size=464951 2024-05-09T20:17:04.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.060+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=5L7C585NA8XZ tag=K7UVR1D9LENL type=write size=3610679 2024-05-09T20:17:04.052 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.060+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~1D90BX47DNQP.0 size=1048576 2024-05-09T20:17:04.054 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.064+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~1D90BX47DNQP.1 size=1048576 2024-05-09T20:17:04.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.064+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~1D90BX47DNQP.2 size=678582 2024-05-09T20:17:04.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.068+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=1D90BX47DNQP tag=04M7MCCZGD4Y type=write size=3824310 2024-05-09T20:17:04.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.072+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=4030KIRLZR8Y type=del size=1397582 2024-05-09T20:17:04.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.072+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=H9Y99HTBL5UI type=del 2024-05-09T20:17:04.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.076+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=ESTSXJW6Y9PG type=write size=360941 2024-05-09T20:17:04.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.080+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=EJK6QPDNQNCD type=del 2024-05-09T20:17:04.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.084+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~IR47Y9UFI5YC.0 size=895580 2024-05-09T20:17:04.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.084+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=IR47Y9UFI5YC tag=MOCDYX3H8XT9 type=write size=1944156 2024-05-09T20:17:04.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.096+0000 7fdc882e58c0 1 < suggested remove operation key=NYHR6GTPQYZS tag=84HKWUPYDTCJ type=del 2024-05-09T20:17:04.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.100+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=AOONV2NB1CVQ type=del size=937242 2024-05-09T20:17:04.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.108+0000 7fdc882e58c0 1 < suggested remove operation key=JS90W6PIREMD tag=30RHFCCWP1S5 type=del 2024-05-09T20:17:04.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.108+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=UK04N37KV0IU type=del size=3501642 2024-05-09T20:17:04.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.112+0000 7fdc882e58c0 1 < canceled part key=_multipart_OGVA2778MJWU.2~099PX5UH3K79.0 size=1048576 2024-05-09T20:17:04.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.112+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~099PX5UH3K79.1 size=319497 2024-05-09T20:17:04.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.116+0000 7fdc882e58c0 1 < canceled multipart upload key=OGVA2778MJWU upload=099PX5UH3K79 tag=D2I0OQQH7ZA3 type=write size=2416649 2024-05-09T20:17:04.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.120+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=J6D0468VA5UV type=write size=70433 2024-05-09T20:17:04.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.124+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~XHI5F2Q0SCQR.0 size=1048576 2024-05-09T20:17:04.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.124+0000 7fdc882e58c0 1 < canceled part key=_multipart_OGVA2778MJWU.2~XHI5F2Q0SCQR.1 size=770139 2024-05-09T20:17:04.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.128+0000 7fdc882e58c0 1 < canceled multipart upload key=OGVA2778MJWU upload=XHI5F2Q0SCQR tag=AGTR7Z3JPDWQ type=write size=2867291 2024-05-09T20:17:04.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.132+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~AX2G430V8LGW.0 size=184226 2024-05-09T20:17:04.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.132+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=AX2G430V8LGW tag=26DIMA9GBM10 type=write size=1232802 2024-05-09T20:17:04.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.136+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=70YQCI6G1PSD type=del 2024-05-09T20:17:04.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.144+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~7Y1KUFBHLZQA.0 size=1048576 2024-05-09T20:17:04.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.144+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~7Y1KUFBHLZQA.1 size=1048576 2024-05-09T20:17:04.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.148+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~7Y1KUFBHLZQA.2 size=928678 2024-05-09T20:17:04.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.148+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=7Y1KUFBHLZQA tag=GTLOHFMNGH3J type=write size=4074406 2024-05-09T20:17:04.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.152+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~33N9Q153OFLM.0 size=1048576 2024-05-09T20:17:04.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.152+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~33N9Q153OFLM.1 size=269084 2024-05-09T20:17:04.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.160+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=33N9Q153OFLM tag=B37R4BZTXN5S type=write size=2366236 2024-05-09T20:17:04.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.168+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~QC9G6CPDR1WF.0 size=1048576 2024-05-09T20:17:04.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.172+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~QC9G6CPDR1WF.1 size=442942 2024-05-09T20:17:04.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.172+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=QC9G6CPDR1WF tag=XMNVS9UOOXAZ type=write size=2540094 2024-05-09T20:17:04.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.176+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=HB6OZFWU3E31 type=del 2024-05-09T20:17:04.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.184+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~PVVHNN02GEL0.0 size=1048576 2024-05-09T20:17:04.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.184+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~PVVHNN02GEL0.1 size=1048576 2024-05-09T20:17:04.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.188+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~PVVHNN02GEL0.2 size=262529 2024-05-09T20:17:04.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.188+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=PVVHNN02GEL0 tag=P4WD15VKDSLU type=write size=3408257 2024-05-09T20:17:04.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.192+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~J0TX0NGS7WF7.0 size=1048576 2024-05-09T20:17:04.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.192+0000 7fdc882e58c0 1 < canceled part key=_multipart_346HV1XQKCOZ.2~J0TX0NGS7WF7.1 size=1048576 2024-05-09T20:17:04.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.196+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~J0TX0NGS7WF7.2 size=529195 2024-05-09T20:17:04.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.204+0000 7fdc882e58c0 1 < canceled multipart upload key=346HV1XQKCOZ upload=J0TX0NGS7WF7 tag=34I5LZOVOYOZ type=write size=3674923 2024-05-09T20:17:04.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.208+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~MYJNK1FWCD8W.0 size=718672 2024-05-09T20:17:04.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.208+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=MYJNK1FWCD8W tag=5SMQGI8N3CMA type=write size=1767248 2024-05-09T20:17:04.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.212+0000 7fdc882e58c0 -1 < failed to complete operation key=5H2GWM9LQELZ tag=TSXHO9R9YRKQ type=del size=1851002: (22) Invalid argument 2024-05-09T20:17:04.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.220+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~ZVK4W34RA5RA.0 size=1048576 2024-05-09T20:17:04.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.224+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~ZVK4W34RA5RA.1 size=857361 2024-05-09T20:17:04.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.224+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=ZVK4W34RA5RA tag=62S1HEJ093GA type=write size=2954513 2024-05-09T20:17:04.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.228+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~VJO7NVDH6KXM.0 size=1048576 2024-05-09T20:17:04.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.228+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~VJO7NVDH6KXM.1 size=707653 2024-05-09T20:17:04.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.236+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=VJO7NVDH6KXM tag=HJHJE5UL7DH9 type=write size=2804805 2024-05-09T20:17:04.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.240+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~K0GYALIV9DOP.0 size=1048576 2024-05-09T20:17:04.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.244+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~K0GYALIV9DOP.1 size=1048576 2024-05-09T20:17:04.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.248+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~K0GYALIV9DOP.2 size=394013 2024-05-09T20:17:04.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.248+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=K0GYALIV9DOP tag=1CL4L94O7TUO type=write size=3539741 2024-05-09T20:17:04.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.248+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=QILFWYTNGP90 type=del 2024-05-09T20:17:04.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.256+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~IWC1OWNEN6HQ.0 size=949214 2024-05-09T20:17:04.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.256+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=IWC1OWNEN6HQ tag=4RMWLZWLENGL type=write size=1997790 2024-05-09T20:17:04.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.260+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=E9V7G5712IW4 type=del 2024-05-09T20:17:04.254 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.264+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=5VMO4OY2YK7A type=del size=3784436 2024-05-09T20:17:04.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.268+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~5L7C585NA8XZ.0 size=1048576 2024-05-09T20:17:04.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.268+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~5L7C585NA8XZ.1 size=1048576 2024-05-09T20:17:04.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.272+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~5L7C585NA8XZ.2 size=464951 2024-05-09T20:17:04.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.276+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=5L7C585NA8XZ tag=K7UVR1D9LENL type=write size=3610679 2024-05-09T20:17:04.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.280+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=8FTI7LDO2J56 type=write size=535938 2024-05-09T20:17:04.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.284+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=4030KIRLZR8Y type=del 2024-05-09T20:17:04.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.288+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=6E69B9Y0M8DQ type=del size=1484047 2024-05-09T20:17:04.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.296+0000 7fdc882e58c0 1 < suggested update operation key=T5EL7X0P450L tag=ESTSXJW6Y9PG type=write size=360941 2024-05-09T20:17:04.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.296+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=JIQJQJL108DG type=del size=3294060 2024-05-09T20:17:04.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.300+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~IR47Y9UFI5YC.0 size=895580 2024-05-09T20:17:04.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.308+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=IR47Y9UFI5YC tag=MOCDYX3H8XT9 type=write size=1944156 2024-05-09T20:17:04.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.312+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=WO6YPLAQ54Q1 type=del size=3201345 2024-05-09T20:17:04.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.316+0000 7fdc882e58c0 1 < canceled operation key=8FFZZDURUWM1 tag=AOONV2NB1CVQ type=del size=937242 2024-05-09T20:17:04.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.316+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=SU2IMFEKZDKR type=write size=665848 2024-05-09T20:17:04.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.320+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=UK04N37KV0IU type=del 2024-05-09T20:17:04.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.324+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=VTW0YW1XGFDP type=del size=1408920 2024-05-09T20:17:04.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.328+0000 7fdc882e58c0 1 < completed write operation key=TQ5ZHATLNVPP tag=J6D0468VA5UV type=write size=70433 2024-05-09T20:17:04.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.332+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=O4JV92OQY8CC type=del size=1684497 2024-05-09T20:17:04.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.336+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~AX2G430V8LGW.0 size=184226 2024-05-09T20:17:04.333 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.340+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=AX2G430V8LGW tag=26DIMA9GBM10 type=write size=1232802 2024-05-09T20:17:04.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.344+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=M2GGYX12N7N2 type=del size=3064710 2024-05-09T20:17:04.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.348+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~7Y1KUFBHLZQA.0 size=1048576 2024-05-09T20:17:04.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.352+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~7Y1KUFBHLZQA.1 size=1048576 2024-05-09T20:17:04.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.356+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~7Y1KUFBHLZQA.2 size=928678 2024-05-09T20:17:04.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.360+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=7Y1KUFBHLZQA tag=GTLOHFMNGH3J type=write size=4074406 2024-05-09T20:17:04.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.364+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=MKEC0FEWARKU type=del size=1847593 2024-05-09T20:17:04.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.364+0000 7fdc882e58c0 1 < canceled part key=_multipart_IZPPFLWVPRKO.2~QC9G6CPDR1WF.0 size=1048576 2024-05-09T20:17:04.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.368+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~QC9G6CPDR1WF.1 size=442942 2024-05-09T20:17:04.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.372+0000 7fdc882e58c0 1 < canceled multipart upload key=IZPPFLWVPRKO upload=QC9G6CPDR1WF tag=XMNVS9UOOXAZ type=write size=2540094 2024-05-09T20:17:04.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.380+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~Z0ZAS3RR08OT.0 size=1048576 2024-05-09T20:17:04.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.384+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~Z0ZAS3RR08OT.1 size=335195 2024-05-09T20:17:04.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.384+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=Z0ZAS3RR08OT tag=8ZC8I24GWA9A type=write size=2432347 2024-05-09T20:17:04.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.388+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~PVVHNN02GEL0.0 size=1048576 2024-05-09T20:17:04.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.388+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~PVVHNN02GEL0.1 size=1048576 2024-05-09T20:17:04.385 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.392+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~PVVHNN02GEL0.2 size=262529 2024-05-09T20:17:04.391 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.400+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=PVVHNN02GEL0 tag=P4WD15VKDSLU type=write size=3408257 2024-05-09T20:17:04.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.404+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=T3J71SNNEIE5 type=write size=7359 2024-05-09T20:17:04.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.408+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~MYJNK1FWCD8W.0 size=718672 2024-05-09T20:17:04.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.416+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=MYJNK1FWCD8W tag=5SMQGI8N3CMA type=write size=1767248 2024-05-09T20:17:04.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~OACGELZK3X3T.0 size=1048576 2024-05-09T20:17:04.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.424+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~OACGELZK3X3T.1 size=87217 2024-05-09T20:17:04.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.424+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=OACGELZK3X3T tag=NC63RUP4I83O type=write size=2184369 2024-05-09T20:17:04.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.428+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~ZVK4W34RA5RA.0 size=1048576 2024-05-09T20:17:04.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.432+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~ZVK4W34RA5RA.1 size=857361 2024-05-09T20:17:04.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.436+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=ZVK4W34RA5RA tag=62S1HEJ093GA type=write size=2954513 2024-05-09T20:17:04.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.440+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=4KA54R2FBL17 type=del size=370371 2024-05-09T20:17:04.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.444+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~K0GYALIV9DOP.0 size=1048576 2024-05-09T20:17:04.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.448+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~K0GYALIV9DOP.1 size=1048576 2024-05-09T20:17:04.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.448+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~K0GYALIV9DOP.2 size=394013 2024-05-09T20:17:04.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.456+0000 7fdc882e58c0 1 < completed multipart upload key=NBAECV9L85UK upload=K0GYALIV9DOP tag=1CL4L94O7TUO type=write size=3539741 2024-05-09T20:17:04.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.460+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=TJ6F01SAIUQP type=del size=3436815 2024-05-09T20:17:04.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.464+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~IWC1OWNEN6HQ.0 size=949214 2024-05-09T20:17:04.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.472+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=IWC1OWNEN6HQ tag=4RMWLZWLENGL type=write size=1997790 2024-05-09T20:17:04.465 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.472+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=Q69JH3OK9PUF type=del size=1854688 2024-05-09T20:17:04.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.476+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=5VMO4OY2YK7A type=del 2024-05-09T20:17:04.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.480+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=Y95AS423QI08 type=del size=1055869 2024-05-09T20:17:04.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.484+0000 7fdc882e58c0 1 < completed write operation key=346HV1XQKCOZ tag=8FTI7LDO2J56 type=write size=535938 2024-05-09T20:17:04.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.488+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=BHQ0HJ15VFM0 type=write size=308387 2024-05-09T20:17:04.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.492+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=6E69B9Y0M8DQ type=del 2024-05-09T20:17:04.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.496+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=G74VXCW9ID6W type=del size=46572 2024-05-09T20:17:04.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.504+0000 7fdc882e58c0 1 < suggested remove operation key=NNGWUSCETMH4 tag=JIQJQJL108DG type=del 2024-05-09T20:17:04.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.508+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=9QNT4EA57LSX type=del size=2864928 2024-05-09T20:17:04.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.512+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=WO6YPLAQ54Q1 type=del 2024-05-09T20:17:04.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.516+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~QM0HV218KH84.0 size=1048576 2024-05-09T20:17:04.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.520+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~QM0HV218KH84.1 size=173232 2024-05-09T20:17:04.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.520+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=QM0HV218KH84 tag=UT7JSIASXUU4 type=write size=2270384 2024-05-09T20:17:04.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.524+0000 7fdc882e58c0 1 < completed write operation key=EK78CNKUHNKO tag=SU2IMFEKZDKR type=write size=665848 2024-05-09T20:17:04.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.532+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~4OIKP5Z2HBR2.0 size=490097 2024-05-09T20:17:04.525 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.532+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=4OIKP5Z2HBR2 tag=IR0MYLLPJ5YR type=write size=1538673 2024-05-09T20:17:04.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.536+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=VTW0YW1XGFDP type=del 2024-05-09T20:17:04.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.540+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=CAH0GZM0D6O3 type=write size=136984 2024-05-09T20:17:04.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.544+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=O4JV92OQY8CC type=del 2024-05-09T20:17:04.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.552+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~YRB9B063JBX5.0 size=800737 2024-05-09T20:17:04.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.552+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=YRB9B063JBX5 tag=HCI24R0DQ726 type=write size=1849313 2024-05-09T20:17:04.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.552+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=M2GGYX12N7N2 type=del 2024-05-09T20:17:04.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.556+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=3G2H3T607QP3 type=del size=3167663 2024-05-09T20:17:04.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.564+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=MKEC0FEWARKU type=del 2024-05-09T20:17:04.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.568+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~WZJ762RPH373.0 size=1048576 2024-05-09T20:17:04.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.572+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~WZJ762RPH373.1 size=1048576 2024-05-09T20:17:04.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.576+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~WZJ762RPH373.2 size=109198 2024-05-09T20:17:04.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.576+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=WZJ762RPH373 tag=CWA95SR02PPP type=write size=3254926 2024-05-09T20:17:04.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.580+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~Z0ZAS3RR08OT.0 size=1048576 2024-05-09T20:17:04.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.580+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~Z0ZAS3RR08OT.1 size=335195 2024-05-09T20:17:04.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.588+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=Z0ZAS3RR08OT tag=8ZC8I24GWA9A type=write size=2432347 2024-05-09T20:17:04.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.592+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~NJ30YDD4SB0V.0 size=1048576 2024-05-09T20:17:04.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.596+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~NJ30YDD4SB0V.1 size=810582 2024-05-09T20:17:04.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.596+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=NJ30YDD4SB0V tag=152ZG2XERTR7 type=write size=2907734 2024-05-09T20:17:04.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.600+0000 7fdc882e58c0 1 < completed write operation key=T6T3IOW9VPA0 tag=T3J71SNNEIE5 type=write size=7359 2024-05-09T20:17:04.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.604+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~B80T8ERP5QT8.0 size=432674 2024-05-09T20:17:04.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.604+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=B80T8ERP5QT8 tag=U0A10J6VLDET type=write size=1481250 2024-05-09T20:17:04.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.608+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~OACGELZK3X3T.0 size=1048576 2024-05-09T20:17:04.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.612+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~OACGELZK3X3T.1 size=87217 2024-05-09T20:17:04.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.616+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=OACGELZK3X3T tag=NC63RUP4I83O type=write size=2184369 2024-05-09T20:17:04.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.620+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=PW4O301OLTY9 type=write size=936033 2024-05-09T20:17:04.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.624+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=4KA54R2FBL17 type=del 2024-05-09T20:17:04.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.632+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~SET81U4JIVQC.0 size=1048576 2024-05-09T20:17:04.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.636+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~SET81U4JIVQC.1 size=750542 2024-05-09T20:17:04.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.636+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=SET81U4JIVQC tag=GIKSRMBO372S type=write size=2847694 2024-05-09T20:17:04.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.636+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=TJ6F01SAIUQP type=del 2024-05-09T20:17:04.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.644+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~3BTP2LJACYZJ.0 size=1048576 2024-05-09T20:17:04.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.648+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~3BTP2LJACYZJ.1 size=1048576 2024-05-09T20:17:04.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.652+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~3BTP2LJACYZJ.2 size=431468 2024-05-09T20:17:04.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.652+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=3BTP2LJACYZJ tag=INM0NYT544LR type=write size=3577196 2024-05-09T20:17:04.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.660+0000 7fdc882e58c0 1 < suggested update operation key=J2Q86HKRMAKE tag=Q69JH3OK9PUF type=del size=1854688 2024-05-09T20:17:04.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.664+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=7C6FNQ223A30 type=del size=2870030 2024-05-09T20:17:04.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.668+0000 7fdc882e58c0 1 < canceled operation key=EK78CNKUHNKO tag=Y95AS423QI08 type=del size=1055869 2024-05-09T20:17:04.664 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.672+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~A94VMJPJ0QOM.0 size=807976 2024-05-09T20:17:04.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.672+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=A94VMJPJ0QOM tag=JIXTE5FWHZQ9 type=write size=1856552 2024-05-09T20:17:04.668 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.676+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=BHQ0HJ15VFM0 type=write size=308387 2024-05-09T20:17:04.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.680+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=SQT8ABRJ8GWH type=del size=4100646 2024-05-09T20:17:04.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.684+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=G74VXCW9ID6W type=del 2024-05-09T20:17:04.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.688+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=U54UQW3OPDHF type=del size=4120856 2024-05-09T20:17:04.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.692+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=9QNT4EA57LSX type=del 2024-05-09T20:17:04.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.700+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~NKDBWQFNSM8I.0 size=1048576 2024-05-09T20:17:04.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.700+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~NKDBWQFNSM8I.1 size=243138 2024-05-09T20:17:04.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.700+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=NKDBWQFNSM8I tag=N68M6K2UQL73 type=write size=2340290 2024-05-09T20:17:04.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.704+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~QM0HV218KH84.0 size=1048576 2024-05-09T20:17:04.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.708+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~QM0HV218KH84.1 size=173232 2024-05-09T20:17:04.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.712+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=QM0HV218KH84 tag=UT7JSIASXUU4 type=write size=2270384 2024-05-09T20:17:04.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.720+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~FO1IHWGXRI6H.0 size=246096 2024-05-09T20:17:04.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.720+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=FO1IHWGXRI6H tag=O4KRMPBMH3V3 type=write size=1294672 2024-05-09T20:17:04.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.724+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~4OIKP5Z2HBR2.0 size=490097 2024-05-09T20:17:04.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.728+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=4OIKP5Z2HBR2 tag=IR0MYLLPJ5YR type=write size=1538673 2024-05-09T20:17:04.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~7MGJLVERQBLL.0 size=1048576 2024-05-09T20:17:04.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.736+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~7MGJLVERQBLL.1 size=987000 2024-05-09T20:17:04.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.736+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=7MGJLVERQBLL tag=2R1A22WKSBBJ type=write size=3084152 2024-05-09T20:17:04.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.740+0000 7fdc882e58c0 1 < completed write operation key=G0DX1F257RLD tag=CAH0GZM0D6O3 type=write size=136984 2024-05-09T20:17:04.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.744+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=SMV4ZF3MTXEP type=del size=2246296 2024-05-09T20:17:04.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.748+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~YRB9B063JBX5.0 size=800737 2024-05-09T20:17:04.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.756+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=YRB9B063JBX5 tag=HCI24R0DQ726 type=write size=1849313 2024-05-09T20:17:04.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.760+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=UYNXEADOSO57 type=del size=212436 2024-05-09T20:17:04.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.764+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=3G2H3T607QP3 type=del 2024-05-09T20:17:04.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.772+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~0775L1Q3BN03.0 size=1048576 2024-05-09T20:17:04.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.772+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~0775L1Q3BN03.1 size=60665 2024-05-09T20:17:04.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.772+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=0775L1Q3BN03 tag=QL4MBNI1GNT5 type=write size=2157817 2024-05-09T20:17:04.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.776+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~WZJ762RPH373.0 size=1048576 2024-05-09T20:17:04.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.780+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~WZJ762RPH373.1 size=1048576 2024-05-09T20:17:04.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.784+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~WZJ762RPH373.2 size=109198 2024-05-09T20:17:04.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.788+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=WZJ762RPH373 tag=CWA95SR02PPP type=write size=3254926 2024-05-09T20:17:04.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.792+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=MRXI2IWF3T66 type=del size=3193826 2024-05-09T20:17:04.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.796+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~NJ30YDD4SB0V.0 size=1048576 2024-05-09T20:17:04.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.800+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~NJ30YDD4SB0V.1 size=810582 2024-05-09T20:17:04.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.804+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=NJ30YDD4SB0V tag=152ZG2XERTR7 type=write size=2907734 2024-05-09T20:17:04.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.808+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=6QT2EZR6NW3J type=del size=246968 2024-05-09T20:17:04.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.812+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~B80T8ERP5QT8.0 size=432674 2024-05-09T20:17:04.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.816+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=B80T8ERP5QT8 tag=U0A10J6VLDET type=write size=1481250 2024-05-09T20:17:04.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.824+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~RPG3IDRWZBOJ.0 size=1048576 2024-05-09T20:17:04.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.824+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~RPG3IDRWZBOJ.1 size=1048576 2024-05-09T20:17:04.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.828+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~RPG3IDRWZBOJ.2 size=882796 2024-05-09T20:17:04.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.828+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=RPG3IDRWZBOJ tag=MOLHQODZHWFP type=write size=4028524 2024-05-09T20:17:04.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.836+0000 7fdc882e58c0 1 < suggested update operation key=WC5YH0OU51WX tag=PW4O301OLTY9 type=write size=936033 2024-05-09T20:17:04.830 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.840+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=NSA9Z9SPRJHN type=del size=485953 2024-05-09T20:17:04.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.840+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~SET81U4JIVQC.0 size=1048576 2024-05-09T20:17:04.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.844+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~SET81U4JIVQC.1 size=750542 2024-05-09T20:17:04.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.852+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=SET81U4JIVQC tag=GIKSRMBO372S type=write size=2847694 2024-05-09T20:17:04.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.852+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=G7176U0NE6E2 type=del size=1158704 2024-05-09T20:17:04.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.856+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~3BTP2LJACYZJ.0 size=1048576 2024-05-09T20:17:04.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.860+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~3BTP2LJACYZJ.1 size=1048576 2024-05-09T20:17:04.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.860+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~3BTP2LJACYZJ.2 size=431468 2024-05-09T20:17:04.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.864+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=3BTP2LJACYZJ tag=INM0NYT544LR type=write size=3577196 2024-05-09T20:17:04.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.868+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=SR0W3U4K8P0J type=del size=1803474 2024-05-09T20:17:04.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.872+0000 7fdc882e58c0 -1 < failed to complete operation key=WC5YH0OU51WX tag=7C6FNQ223A30 type=del size=2870030: (22) Invalid argument 2024-05-09T20:17:04.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.876+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=QRRU6CVYY8U5 type=del size=2294465 2024-05-09T20:17:04.870 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.880+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~A94VMJPJ0QOM.0 size=807976 2024-05-09T20:17:04.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.884+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=A94VMJPJ0QOM tag=JIXTE5FWHZQ9 type=write size=1856552 2024-05-09T20:17:04.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.888+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=U7BNAKDN64C1 type=del size=3285250 2024-05-09T20:17:04.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.888+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=SQT8ABRJ8GWH type=del 2024-05-09T20:17:04.884 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.892+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=YT551FWQVX5C type=del size=1146489 2024-05-09T20:17:04.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.900+0000 7fdc882e58c0 1 < suggested update operation key=346HV1XQKCOZ tag=U54UQW3OPDHF type=del size=4120856 2024-05-09T20:17:04.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.904+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~O6GBEVXP7T3J.0 size=1048576 2024-05-09T20:17:04.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.908+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~O6GBEVXP7T3J.1 size=187688 2024-05-09T20:17:04.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.908+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=O6GBEVXP7T3J tag=WU44L36JBW1K type=write size=2284840 2024-05-09T20:17:04.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.908+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~NKDBWQFNSM8I.0 size=1048576 2024-05-09T20:17:04.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.912+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~NKDBWQFNSM8I.1 size=243138 2024-05-09T20:17:04.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.916+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=NKDBWQFNSM8I tag=N68M6K2UQL73 type=write size=2340290 2024-05-09T20:17:04.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.920+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~Q23Y84DSGELN.0 size=542719 2024-05-09T20:17:04.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.920+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=Q23Y84DSGELN tag=2XR735WIWSRN type=write size=1591295 2024-05-09T20:17:04.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.924+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~FO1IHWGXRI6H.0 size=246096 2024-05-09T20:17:04.919 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.928+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=FO1IHWGXRI6H tag=O4KRMPBMH3V3 type=write size=1294672 2024-05-09T20:17:04.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.932+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~0PPN5CH4ZLWM.0 size=1048576 2024-05-09T20:17:04.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~0PPN5CH4ZLWM.1 size=1048576 2024-05-09T20:17:04.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.936+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~0PPN5CH4ZLWM.2 size=602107 2024-05-09T20:17:04.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.936+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=0PPN5CH4ZLWM tag=U1F31SI8D4PX type=write size=3747835 2024-05-09T20:17:04.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.940+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~7MGJLVERQBLL.0 size=1048576 2024-05-09T20:17:04.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.944+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~7MGJLVERQBLL.1 size=987000 2024-05-09T20:17:04.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.948+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=7MGJLVERQBLL tag=2R1A22WKSBBJ type=write size=3084152 2024-05-09T20:17:04.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.952+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=BSNB5TDVN35T type=del size=296467 2024-05-09T20:17:04.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.956+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=SMV4ZF3MTXEP type=del 2024-05-09T20:17:04.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.959+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=QFHOML6SNUMC type=del size=18130 2024-05-09T20:17:04.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.963+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=UYNXEADOSO57 type=del 2024-05-09T20:17:04.958 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.967+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=OHD4M9RLKBVZ type=del size=2698016 2024-05-09T20:17:04.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.967+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~0775L1Q3BN03.0 size=1048576 2024-05-09T20:17:04.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.971+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~0775L1Q3BN03.1 size=60665 2024-05-09T20:17:04.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.979+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=0775L1Q3BN03 tag=QL4MBNI1GNT5 type=write size=2157817 2024-05-09T20:17:04.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.983+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~EHYPJ9CHSW1P.0 size=468744 2024-05-09T20:17:04.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.983+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=EHYPJ9CHSW1P tag=BKFSSK6W4GXL type=write size=1517320 2024-05-09T20:17:04.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.991+0000 7fdc882e58c0 1 < suggested update operation key=346HV1XQKCOZ tag=MRXI2IWF3T66 type=del size=3193826 2024-05-09T20:17:04.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:04.999+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~PUKJ23VKZW9X.0 size=1048576 2024-05-09T20:17:04.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.003+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~PUKJ23VKZW9X.1 size=1048576 2024-05-09T20:17:04.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.003+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~PUKJ23VKZW9X.2 size=730253 2024-05-09T20:17:04.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.003+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=PUKJ23VKZW9X tag=REUK9MSYTVBP type=write size=3875981 2024-05-09T20:17:05.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.011+0000 7fdc882e58c0 1 < suggested update operation key=G0DX1F257RLD tag=6QT2EZR6NW3J type=del size=246968 2024-05-09T20:17:05.008 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.015+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=Y0UHFMJOF65L type=del size=2632398 2024-05-09T20:17:05.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.019+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~RPG3IDRWZBOJ.0 size=1048576 2024-05-09T20:17:05.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.023+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~RPG3IDRWZBOJ.1 size=1048576 2024-05-09T20:17:05.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.027+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~RPG3IDRWZBOJ.2 size=882796 2024-05-09T20:17:05.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.031+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=RPG3IDRWZBOJ tag=MOLHQODZHWFP type=write size=4028524 2024-05-09T20:17:05.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.035+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=LWQMWR7U5ISL type=del size=161154 2024-05-09T20:17:05.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.039+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=NSA9Z9SPRJHN type=del 2024-05-09T20:17:05.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.047+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~DHN2PNKR8XNP.0 size=1048576 2024-05-09T20:17:05.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.047+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~DHN2PNKR8XNP.1 size=1048576 2024-05-09T20:17:05.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.051+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~DHN2PNKR8XNP.2 size=774600 2024-05-09T20:17:05.044 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.051+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=DHN2PNKR8XNP tag=JHCBOCLM1JAZ type=write size=3920328 2024-05-09T20:17:05.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.055+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=G7176U0NE6E2 type=del 2024-05-09T20:17:05.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.063+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=JQXNUTYRJ6IL type=write size=807766 2024-05-09T20:17:05.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.067+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=SR0W3U4K8P0J type=del 2024-05-09T20:17:05.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.071+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=YNQADY9143QV type=write size=378590 2024-05-09T20:17:05.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.075+0000 7fdc882e58c0 1 < canceled operation key=FNB9B18LIA8L tag=QRRU6CVYY8U5 type=del size=2294465 2024-05-09T20:17:05.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.079+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~LU3LAJIRC6QH.0 size=1048576 2024-05-09T20:17:05.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.083+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~LU3LAJIRC6QH.1 size=180252 2024-05-09T20:17:05.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.083+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=LU3LAJIRC6QH tag=BSYI0XRJQTTI type=write size=2277404 2024-05-09T20:17:05.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.087+0000 7fdc882e58c0 -1 < failed to complete operation key=G0DX1F257RLD tag=U7BNAKDN64C1 type=del size=3285250: (22) Invalid argument 2024-05-09T20:17:05.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.091+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~50UX5P0DUJIR.0 size=84256 2024-05-09T20:17:05.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.091+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=50UX5P0DUJIR tag=EBQAHFPV7B38 type=write size=1132832 2024-05-09T20:17:05.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.095+0000 7fdc882e58c0 -1 < failed to complete operation key=346HV1XQKCOZ tag=YT551FWQVX5C type=del size=1146489: (22) Invalid argument 2024-05-09T20:17:05.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.099+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=9J0OES8TNVZ4 type=write size=1025331 2024-05-09T20:17:05.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.103+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~O6GBEVXP7T3J.0 size=1048576 2024-05-09T20:17:05.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.107+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~O6GBEVXP7T3J.1 size=187688 2024-05-09T20:17:05.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.111+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=O6GBEVXP7T3J tag=WU44L36JBW1K type=write size=2284840 2024-05-09T20:17:05.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.115+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=P8GYKXMPEDBG type=del size=2475828 2024-05-09T20:17:05.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.119+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~Q23Y84DSGELN.0 size=542719 2024-05-09T20:17:05.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.123+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=Q23Y84DSGELN tag=2XR735WIWSRN type=write size=1591295 2024-05-09T20:17:05.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.127+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=3WUF7XSC8J9A type=del size=2201234 2024-05-09T20:17:05.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.131+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~0PPN5CH4ZLWM.0 size=1048576 2024-05-09T20:17:05.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.135+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~0PPN5CH4ZLWM.1 size=1048576 2024-05-09T20:17:05.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.139+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~0PPN5CH4ZLWM.2 size=602107 2024-05-09T20:17:05.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.143+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=0PPN5CH4ZLWM tag=U1F31SI8D4PX type=write size=3747835 2024-05-09T20:17:05.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.147+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=5JAGQUBZWRMK type=del size=2946675 2024-05-09T20:17:05.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.151+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=BSNB5TDVN35T type=del 2024-05-09T20:17:05.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.155+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=WQS5OZUILO1Y type=del size=1688460 2024-05-09T20:17:05.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.159+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=QFHOML6SNUMC type=del 2024-05-09T20:17:05.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.163+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=ACVHUOHIG3KH type=del size=374330 2024-05-09T20:17:05.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.167+0000 7fdc882e58c0 1 < canceled operation key=NBAECV9L85UK tag=OHD4M9RLKBVZ type=del size=2698016 2024-05-09T20:17:05.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.171+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=19OICNK99RL7 type=write size=11367 2024-05-09T20:17:05.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.171+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~EHYPJ9CHSW1P.0 size=468744 2024-05-09T20:17:05.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.179+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=EHYPJ9CHSW1P tag=BKFSSK6W4GXL type=write size=1517320 2024-05-09T20:17:05.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.187+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~OU6ARMTG2KCW.0 size=1048576 2024-05-09T20:17:05.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.187+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~OU6ARMTG2KCW.1 size=1048576 2024-05-09T20:17:05.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.191+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~OU6ARMTG2KCW.2 size=635075 2024-05-09T20:17:05.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.191+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=OU6ARMTG2KCW tag=IS6XR82U4KBN type=write size=3780803 2024-05-09T20:17:05.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.195+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~PUKJ23VKZW9X.0 size=1048576 2024-05-09T20:17:05.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.199+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~PUKJ23VKZW9X.1 size=1048576 2024-05-09T20:17:05.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.199+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~PUKJ23VKZW9X.2 size=730253 2024-05-09T20:17:05.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.207+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=PUKJ23VKZW9X tag=REUK9MSYTVBP type=write size=3875981 2024-05-09T20:17:05.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.211+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~UYJ1KZF6SZEH.0 size=1048576 2024-05-09T20:17:05.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.215+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~UYJ1KZF6SZEH.1 size=855447 2024-05-09T20:17:05.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.215+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=UYJ1KZF6SZEH tag=V674OMW0C253 type=write size=2952599 2024-05-09T20:17:05.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.223+0000 7fdc882e58c0 1 < suggested remove operation key=EK78CNKUHNKO tag=Y0UHFMJOF65L type=del 2024-05-09T20:17:05.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.227+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~5EE0YSPB1I9H.0 size=1048576 2024-05-09T20:17:05.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.231+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~5EE0YSPB1I9H.1 size=1024182 2024-05-09T20:17:05.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.231+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=5EE0YSPB1I9H tag=QN98E0FQ5QA0 type=write size=3121334 2024-05-09T20:17:05.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.235+0000 7fdc882e58c0 1 < canceled operation key=EIMPFFK4I6NZ tag=LWQMWR7U5ISL type=del size=161154 2024-05-09T20:17:05.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.239+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=OXP759GMQ34X type=del size=2093445 2024-05-09T20:17:05.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.243+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~DHN2PNKR8XNP.0 size=1048576 2024-05-09T20:17:05.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.243+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~DHN2PNKR8XNP.1 size=1048576 2024-05-09T20:17:05.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.247+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~DHN2PNKR8XNP.2 size=774600 2024-05-09T20:17:05.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.255+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=DHN2PNKR8XNP tag=JHCBOCLM1JAZ type=write size=3920328 2024-05-09T20:17:05.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.259+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=VIC7W8YT7EPX type=write size=753559 2024-05-09T20:17:05.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.263+0000 7fdc882e58c0 1 < completed write operation key=IZPPFLWVPRKO tag=JQXNUTYRJ6IL type=write size=807766 2024-05-09T20:17:05.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.271+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~TPIQ6B655PUY.0 size=1048576 2024-05-09T20:17:05.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.271+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~TPIQ6B655PUY.1 size=1048576 2024-05-09T20:17:05.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.275+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~TPIQ6B655PUY.2 size=936693 2024-05-09T20:17:05.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.275+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=TPIQ6B655PUY tag=8Q23CAO60Y0Y type=write size=4082421 2024-05-09T20:17:05.272 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.279+0000 7fdc882e58c0 1 < completed write operation key=O07UHAIJZF0R tag=YNQADY9143QV type=write size=378590 2024-05-09T20:17:05.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.283+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=9OYART1UA6QH type=write size=290284 2024-05-09T20:17:05.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.287+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~LU3LAJIRC6QH.0 size=1048576 2024-05-09T20:17:05.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.291+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~LU3LAJIRC6QH.1 size=180252 2024-05-09T20:17:05.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.299+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=LU3LAJIRC6QH tag=BSYI0XRJQTTI type=write size=2277404 2024-05-09T20:17:05.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.303+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~1H4ODBPRXMXH.0 size=1048576 2024-05-09T20:17:05.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.307+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~1H4ODBPRXMXH.1 size=912176 2024-05-09T20:17:05.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.307+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=1H4ODBPRXMXH tag=F3KWJCMRP2RV type=write size=3009328 2024-05-09T20:17:05.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.311+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~50UX5P0DUJIR.0 size=84256 2024-05-09T20:17:05.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.315+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=50UX5P0DUJIR tag=EBQAHFPV7B38 type=write size=1132832 2024-05-09T20:17:05.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.323+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~3F71H9JOAPPN.0 size=1048576 2024-05-09T20:17:05.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.327+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~3F71H9JOAPPN.1 size=1048576 2024-05-09T20:17:05.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.327+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~3F71H9JOAPPN.2 size=686354 2024-05-09T20:17:05.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.327+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=3F71H9JOAPPN tag=FX9DL7D6AQ0C type=write size=3832082 2024-05-09T20:17:05.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.331+0000 7fdc882e58c0 1 < completed write operation key=G0DX1F257RLD tag=9J0OES8TNVZ4 type=write size=1025331 2024-05-09T20:17:05.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.335+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=5YPGC6BZICU4 type=del size=3202475 2024-05-09T20:17:05.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.339+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=P8GYKXMPEDBG type=del 2024-05-09T20:17:05.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.343+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=F5XBFBQRW7L4 type=del size=2787593 2024-05-09T20:17:05.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.347+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=3WUF7XSC8J9A type=del 2024-05-09T20:17:05.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.351+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=LVDYP5EE8UU1 type=write size=199736 2024-05-09T20:17:05.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.355+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=5JAGQUBZWRMK type=del 2024-05-09T20:17:05.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.363+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~RT0AL50J0GTH.0 size=1048576 2024-05-09T20:17:05.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.363+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~RT0AL50J0GTH.1 size=844725 2024-05-09T20:17:05.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.363+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=RT0AL50J0GTH tag=QBKJ7617SNY7 type=write size=2941877 2024-05-09T20:17:05.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.367+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=WQS5OZUILO1Y type=del 2024-05-09T20:17:05.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.371+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=S8GSO5TBXHJ2 type=del size=128874 2024-05-09T20:17:05.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.375+0000 7fdc882e58c0 -1 < failed to complete operation key=EK78CNKUHNKO tag=ACVHUOHIG3KH type=del size=374330: (22) Invalid argument 2024-05-09T20:17:05.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.379+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=BOFV98JIS3SS type=write size=695547 2024-05-09T20:17:05.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.387+0000 7fdc882e58c0 1 < completed write operation key=BS0T1OQOBS67 tag=19OICNK99RL7 type=write size=11367 2024-05-09T20:17:05.384 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.391+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~QR3G5S9ADFQE.0 size=32604 2024-05-09T20:17:05.384 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.391+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=QR3G5S9ADFQE tag=Y5GV8EVP78IM type=write size=1081180 2024-05-09T20:17:05.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.395+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~OU6ARMTG2KCW.0 size=1048576 2024-05-09T20:17:05.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.395+0000 7fdc882e58c0 1 < canceled part key=_multipart_G0DX1F257RLD.2~OU6ARMTG2KCW.1 size=1048576 2024-05-09T20:17:05.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.403+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~OU6ARMTG2KCW.2 size=635075 2024-05-09T20:17:05.400 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.407+0000 7fdc882e58c0 1 < canceled multipart upload key=G0DX1F257RLD upload=OU6ARMTG2KCW tag=IS6XR82U4KBN type=write size=3780803 2024-05-09T20:17:05.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.415+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~G0L517OS5G1Y.0 size=55654 2024-05-09T20:17:05.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.415+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=G0L517OS5G1Y tag=ILAAWMKFMAPX type=write size=1104230 2024-05-09T20:17:05.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.419+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~UYJ1KZF6SZEH.0 size=1048576 2024-05-09T20:17:05.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.423+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~UYJ1KZF6SZEH.1 size=855447 2024-05-09T20:17:05.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.427+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=UYJ1KZF6SZEH tag=V674OMW0C253 type=write size=2952599 2024-05-09T20:17:05.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.435+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~4SZ4V4US6AGR.0 size=599591 2024-05-09T20:17:05.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.435+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=4SZ4V4US6AGR tag=28ODKSELI37I type=write size=1648167 2024-05-09T20:17:05.430 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.439+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~5EE0YSPB1I9H.0 size=1048576 2024-05-09T20:17:05.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.439+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~5EE0YSPB1I9H.1 size=1024182 2024-05-09T20:17:05.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.447+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=5EE0YSPB1I9H tag=QN98E0FQ5QA0 type=write size=3121334 2024-05-09T20:17:05.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.447+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=ISCMZK7KS43T type=del size=2972566 2024-05-09T20:17:05.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.455+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=OXP759GMQ34X type=del size=2093445 2024-05-09T20:17:05.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.459+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=8EASU18FQ0D0 type=write size=797947 2024-05-09T20:17:05.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.467+0000 7fdc882e58c0 1 < suggested remove operation key=WC5YH0OU51WX tag=VIC7W8YT7EPX type=write 2024-05-09T20:17:05.465 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.471+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=HVUK29G1L2IA type=del size=2023082 2024-05-09T20:17:05.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.475+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~TPIQ6B655PUY.0 size=1048576 2024-05-09T20:17:05.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.479+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~TPIQ6B655PUY.1 size=1048576 2024-05-09T20:17:05.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.483+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~TPIQ6B655PUY.2 size=936693 2024-05-09T20:17:05.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.487+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=TPIQ6B655PUY tag=8Q23CAO60Y0Y type=write size=4082421 2024-05-09T20:17:05.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.491+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=HIKNLHLAECS7 type=del size=3452174 2024-05-09T20:17:05.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.499+0000 7fdc882e58c0 1 < suggested update operation key=J2Q86HKRMAKE tag=9OYART1UA6QH type=write size=290284 2024-05-09T20:17:05.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.503+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=D3LZBSYXUW67 type=del size=894077 2024-05-09T20:17:05.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.507+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~1H4ODBPRXMXH.0 size=1048576 2024-05-09T20:17:05.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.511+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~1H4ODBPRXMXH.1 size=912176 2024-05-09T20:17:05.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.515+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=1H4ODBPRXMXH tag=F3KWJCMRP2RV type=write size=3009328 2024-05-09T20:17:05.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.523+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~6FHN4GWK6YSL.0 size=1048576 2024-05-09T20:17:05.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.523+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~6FHN4GWK6YSL.1 size=636235 2024-05-09T20:17:05.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.523+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=6FHN4GWK6YSL tag=VLQZ2F0RXBRQ type=write size=2733387 2024-05-09T20:17:05.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.527+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~3F71H9JOAPPN.0 size=1048576 2024-05-09T20:17:05.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.527+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~3F71H9JOAPPN.1 size=1048576 2024-05-09T20:17:05.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.531+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~3F71H9JOAPPN.2 size=686354 2024-05-09T20:17:05.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.539+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=3F71H9JOAPPN tag=FX9DL7D6AQ0C type=write size=3832082 2024-05-09T20:17:05.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.543+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~CKZ5IRX97JDQ.0 size=1048576 2024-05-09T20:17:05.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.547+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~CKZ5IRX97JDQ.1 size=1048576 2024-05-09T20:17:05.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.547+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~CKZ5IRX97JDQ.2 size=68389 2024-05-09T20:17:05.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.547+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=CKZ5IRX97JDQ tag=232QVTT5T12A type=write size=3214117 2024-05-09T20:17:05.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.555+0000 7fdc882e58c0 1 < suggested update operation key=NNGWUSCETMH4 tag=5YPGC6BZICU4 type=del size=3202475 2024-05-09T20:17:05.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.559+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=JF67VFQZDXGH type=write size=116357 2024-05-09T20:17:05.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.563+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=F5XBFBQRW7L4 type=del 2024-05-09T20:17:05.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.567+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=PB510K7RACJ0 type=del size=3076716 2024-05-09T20:17:05.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.567+0000 7fdc882e58c0 1 < completed write operation key=O07UHAIJZF0R tag=LVDYP5EE8UU1 type=write size=199736 2024-05-09T20:17:05.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.571+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=4WI5LM3SJFLF type=del size=2094353 2024-05-09T20:17:05.568 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.575+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~RT0AL50J0GTH.0 size=1048576 2024-05-09T20:17:05.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.579+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~RT0AL50J0GTH.1 size=844725 2024-05-09T20:17:05.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.583+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=RT0AL50J0GTH tag=QBKJ7617SNY7 type=write size=2941877 2024-05-09T20:17:05.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.587+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=3PBVKNC8EFOA type=del size=230267 2024-05-09T20:17:05.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.591+0000 7fdc882e58c0 -1 < failed to complete operation key=NNGWUSCETMH4 tag=S8GSO5TBXHJ2 type=del size=128874: (22) Invalid argument 2024-05-09T20:17:05.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.599+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~3HULSU9LS3XO.0 size=1048576 2024-05-09T20:17:05.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.599+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~3HULSU9LS3XO.1 size=914464 2024-05-09T20:17:05.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.599+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=3HULSU9LS3XO tag=0DDUVQO86TZD type=write size=3011616 2024-05-09T20:17:05.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.607+0000 7fdc882e58c0 1 < suggested remove operation key=346HV1XQKCOZ tag=BOFV98JIS3SS type=write 2024-05-09T20:17:05.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.611+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~FURM2XUFXIRN.0 size=877682 2024-05-09T20:17:05.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.611+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=FURM2XUFXIRN tag=UYAOE5GL9ELE type=write size=1926258 2024-05-09T20:17:05.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.615+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~QR3G5S9ADFQE.0 size=32604 2024-05-09T20:17:05.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.623+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=QR3G5S9ADFQE tag=Y5GV8EVP78IM type=write size=1081180 2024-05-09T20:17:05.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.627+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~GQ9BOD23FWLV.0 size=1048576 2024-05-09T20:17:05.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.631+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~GQ9BOD23FWLV.1 size=68013 2024-05-09T20:17:05.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.631+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=GQ9BOD23FWLV tag=OETZMU7CQ5PZ type=write size=2165165 2024-05-09T20:17:05.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.631+0000 7fdc882e58c0 1 < canceled part key=_multipart_HPK2T0629B1P.2~G0L517OS5G1Y.0 size=55654 2024-05-09T20:17:05.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.635+0000 7fdc882e58c0 1 < canceled multipart upload key=HPK2T0629B1P upload=G0L517OS5G1Y tag=ILAAWMKFMAPX type=write size=1104230 2024-05-09T20:17:05.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.639+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=A67BWT6R22GG type=write size=200048 2024-05-09T20:17:05.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.639+0000 7fdc882e58c0 1 < canceled part key=_multipart_T5EL7X0P450L.2~4SZ4V4US6AGR.0 size=599591 2024-05-09T20:17:05.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.647+0000 7fdc882e58c0 1 < canceled multipart upload key=T5EL7X0P450L upload=4SZ4V4US6AGR tag=28ODKSELI37I type=write size=1648167 2024-05-09T20:17:05.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.651+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=LDJMYRA56WRV type=del size=3967541 2024-05-09T20:17:05.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.655+0000 7fdc882e58c0 1 < suggested remove operation key=T5EL7X0P450L tag=ISCMZK7KS43T type=del 2024-05-09T20:17:05.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.659+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=4SY4YV1ZPJ2F type=del size=4155781 2024-05-09T20:17:05.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.667+0000 7fdc882e58c0 1 < suggested update operation key=HPK2T0629B1P tag=8EASU18FQ0D0 type=write size=797947 2024-05-09T20:17:05.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.671+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=0CELH07ZNPBY type=del size=3608588 2024-05-09T20:17:05.667 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.675+0000 7fdc882e58c0 -1 < failed to complete operation key=J2Q86HKRMAKE tag=HVUK29G1L2IA type=del size=2023082: (22) Invalid argument 2024-05-09T20:17:05.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.679+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~2QVEKHAQ9BHT.0 size=1048576 2024-05-09T20:17:05.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.683+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~2QVEKHAQ9BHT.1 size=932691 2024-05-09T20:17:05.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.683+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=2QVEKHAQ9BHT tag=GQ0OHV9LY64O type=write size=3029843 2024-05-09T20:17:05.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.691+0000 7fdc882e58c0 1 < suggested update operation key=VZ7XOAT899GU tag=HIKNLHLAECS7 type=del size=3452174 2024-05-09T20:17:05.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.691+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=7V95O6XMASKL type=del size=3144079 2024-05-09T20:17:05.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.695+0000 7fdc882e58c0 1 < canceled operation key=5UMJFJRKUPKR tag=D3LZBSYXUW67 type=del size=894077 2024-05-09T20:17:05.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.699+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=J17LSEM7Y2CR type=del size=1875742 2024-05-09T20:17:05.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.703+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~6FHN4GWK6YSL.0 size=1048576 2024-05-09T20:17:05.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.707+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~6FHN4GWK6YSL.1 size=636235 2024-05-09T20:17:05.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.711+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=6FHN4GWK6YSL tag=VLQZ2F0RXBRQ type=write size=2733387 2024-05-09T20:17:05.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.715+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=1MHL59S45JVK type=del size=2212685 2024-05-09T20:17:05.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.719+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~CKZ5IRX97JDQ.0 size=1048576 2024-05-09T20:17:05.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.723+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~CKZ5IRX97JDQ.1 size=1048576 2024-05-09T20:17:05.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.723+0000 7fdc882e58c0 1 < canceled part key=_multipart_SNPNT5JFJOUS.2~CKZ5IRX97JDQ.2 size=68389 2024-05-09T20:17:05.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.731+0000 7fdc882e58c0 1 < canceled multipart upload key=SNPNT5JFJOUS upload=CKZ5IRX97JDQ tag=232QVTT5T12A type=write size=3214117 2024-05-09T20:17:05.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.735+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=53J9XSAW0ZQJ type=del size=1302184 2024-05-09T20:17:05.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.739+0000 7fdc882e58c0 1 < completed write operation key=NYHR6GTPQYZS tag=JF67VFQZDXGH type=write size=116357 2024-05-09T20:17:05.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.747+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~Z7ZB2EM6SJJ3.0 size=1048576 2024-05-09T20:17:05.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.747+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~Z7ZB2EM6SJJ3.1 size=565637 2024-05-09T20:17:05.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.747+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=Z7ZB2EM6SJJ3 tag=E0640AX47TF6 type=write size=2662789 2024-05-09T20:17:05.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.755+0000 7fdc882e58c0 1 < suggested remove operation key=NBAECV9L85UK tag=PB510K7RACJ0 type=del 2024-05-09T20:17:05.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.763+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~O3KJ5902LUIW.0 size=1048576 2024-05-09T20:17:05.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.763+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~O3KJ5902LUIW.1 size=1048576 2024-05-09T20:17:05.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.767+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~O3KJ5902LUIW.2 size=883384 2024-05-09T20:17:05.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.767+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=O3KJ5902LUIW tag=W30D14RK19EN type=write size=4029112 2024-05-09T20:17:05.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.771+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=4WI5LM3SJFLF type=del 2024-05-09T20:17:05.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.775+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=8YN0NP648K7B type=del size=1858608 2024-05-09T20:17:05.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.775+0000 7fdc882e58c0 -1 < failed to complete operation key=T5EL7X0P450L tag=3PBVKNC8EFOA type=del size=230267: (22) Invalid argument 2024-05-09T20:17:05.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.779+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=EXA5HX7B3N7Y type=del size=1323073 2024-05-09T20:17:05.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.783+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~3HULSU9LS3XO.0 size=1048576 2024-05-09T20:17:05.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.783+0000 7fdc882e58c0 1 < canceled part key=_multipart_J2Q86HKRMAKE.2~3HULSU9LS3XO.1 size=914464 2024-05-09T20:17:05.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.787+0000 7fdc882e58c0 1 < canceled multipart upload key=J2Q86HKRMAKE upload=3HULSU9LS3XO tag=0DDUVQO86TZD type=write size=3011616 2024-05-09T20:17:05.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.791+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=VUIG6ZYT3OA2 type=write size=3089 2024-05-09T20:17:05.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.795+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~FURM2XUFXIRN.0 size=877682 2024-05-09T20:17:05.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.803+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=FURM2XUFXIRN tag=UYAOE5GL9ELE type=write size=1926258 2024-05-09T20:17:05.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.803+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=C9HG43MZRQCO type=del size=2236682 2024-05-09T20:17:05.801 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.807+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~GQ9BOD23FWLV.0 size=1048576 2024-05-09T20:17:05.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.811+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~GQ9BOD23FWLV.1 size=68013 2024-05-09T20:17:05.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.815+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=GQ9BOD23FWLV tag=OETZMU7CQ5PZ type=write size=2165165 2024-05-09T20:17:05.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.819+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~QKK1UA6903CH.0 size=1048576 2024-05-09T20:17:05.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.823+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~QKK1UA6903CH.1 size=804771 2024-05-09T20:17:05.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.823+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=QKK1UA6903CH tag=OZPZHS6KUFXA type=write size=2901923 2024-05-09T20:17:05.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.827+0000 7fdc882e58c0 1 < completed write operation key=5H2GWM9LQELZ tag=A67BWT6R22GG type=write size=200048 2024-05-09T20:17:05.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.831+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~M6MRIF1KUGTW.0 size=1048576 2024-05-09T20:17:05.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.831+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~M6MRIF1KUGTW.1 size=1048576 2024-05-09T20:17:05.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.835+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~M6MRIF1KUGTW.2 size=314072 2024-05-09T20:17:05.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.835+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=M6MRIF1KUGTW tag=VF83SFGJGI2N type=write size=3459800 2024-05-09T20:17:05.830 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.839+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=LDJMYRA56WRV type=del 2024-05-09T20:17:05.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.843+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~LWW2FWZ1KYFH.0 size=1048576 2024-05-09T20:17:05.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.843+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~LWW2FWZ1KYFH.1 size=906072 2024-05-09T20:17:05.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.843+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=LWW2FWZ1KYFH tag=JMLA430VTJGZ type=write size=3003224 2024-05-09T20:17:05.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.847+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=4SY4YV1ZPJ2F type=del 2024-05-09T20:17:05.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.847+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=6L9DUOIQX5XU type=del size=2585539 2024-05-09T20:17:05.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.851+0000 7fdc882e58c0 1 < suggested remove operation key=5UMJFJRKUPKR tag=0CELH07ZNPBY type=del 2024-05-09T20:17:05.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.855+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=EZSOMX0J30FG type=del size=3423879 2024-05-09T20:17:05.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.855+0000 7fdc882e58c0 1 < canceled part key=_multipart_5UMJFJRKUPKR.2~2QVEKHAQ9BHT.0 size=1048576 2024-05-09T20:17:05.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.859+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~2QVEKHAQ9BHT.1 size=932691 2024-05-09T20:17:05.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.863+0000 7fdc882e58c0 1 < canceled multipart upload key=5UMJFJRKUPKR upload=2QVEKHAQ9BHT tag=GQ0OHV9LY64O type=write size=3029843 2024-05-09T20:17:05.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.863+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=21TIVMV2QY5V type=del size=2750384 2024-05-09T20:17:05.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.867+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=7V95O6XMASKL type=del 2024-05-09T20:17:05.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.871+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~MXGE09BHXZ9M.0 size=1048576 2024-05-09T20:17:05.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.875+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~MXGE09BHXZ9M.1 size=1048576 2024-05-09T20:17:05.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.875+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~MXGE09BHXZ9M.2 size=469429 2024-05-09T20:17:05.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.875+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=MXGE09BHXZ9M tag=0N5DMPEJ527S type=write size=3615157 2024-05-09T20:17:05.871 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.879+0000 7fdc882e58c0 1 < canceled operation key=T5EL7X0P450L tag=J17LSEM7Y2CR type=del size=1875742 2024-05-09T20:17:05.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.879+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=JHD5S8X1EO4A type=del size=2007120 2024-05-09T20:17:05.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.883+0000 7fdc882e58c0 1 < canceled operation key=BS0T1OQOBS67 tag=1MHL59S45JVK type=del size=2212685 2024-05-09T20:17:05.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.887+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=5SGY03QU5JUE type=del size=1697794 2024-05-09T20:17:05.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.887+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=53J9XSAW0ZQJ type=del 2024-05-09T20:17:05.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.891+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=GH6L20B6MRUK type=write size=442253 2024-05-09T20:17:05.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.891+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~Z7ZB2EM6SJJ3.0 size=1048576 2024-05-09T20:17:05.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.895+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~Z7ZB2EM6SJJ3.1 size=565637 2024-05-09T20:17:05.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.899+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=Z7ZB2EM6SJJ3 tag=E0640AX47TF6 type=write size=2662789 2024-05-09T20:17:05.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.899+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=TJ66XP8VFF9P type=del size=1933174 2024-05-09T20:17:05.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.903+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~O3KJ5902LUIW.0 size=1048576 2024-05-09T20:17:05.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.903+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~O3KJ5902LUIW.1 size=1048576 2024-05-09T20:17:05.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.907+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~O3KJ5902LUIW.2 size=883384 2024-05-09T20:17:05.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.911+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=O3KJ5902LUIW tag=W30D14RK19EN type=write size=4029112 2024-05-09T20:17:05.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.911+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=6E6JWNX6F9QI type=del size=4079972 2024-05-09T20:17:05.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.915+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=8YN0NP648K7B type=del 2024-05-09T20:17:05.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.919+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~2VG7XJXOBL70.0 size=1048576 2024-05-09T20:17:05.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.919+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~2VG7XJXOBL70.1 size=1048576 2024-05-09T20:17:05.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.923+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~2VG7XJXOBL70.2 size=492586 2024-05-09T20:17:05.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.923+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=2VG7XJXOBL70 tag=VOE41A62U5SR type=write size=3638314 2024-05-09T20:17:05.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.927+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=EXA5HX7B3N7Y type=del 2024-05-09T20:17:05.922 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.931+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~C313DHJD9LIT.0 size=182079 2024-05-09T20:17:05.922 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.931+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=C313DHJD9LIT tag=MQWBU9IA9TSW type=write size=1230655 2024-05-09T20:17:05.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.931+0000 7fdc882e58c0 1 < completed write operation key=346HV1XQKCOZ tag=VUIG6ZYT3OA2 type=write size=3089 2024-05-09T20:17:05.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.935+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=YBLRC5Q6135Z type=write size=601564 2024-05-09T20:17:05.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.935+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=C9HG43MZRQCO type=del 2024-05-09T20:17:05.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.939+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~Q79CHOE8K9VU.0 size=410 2024-05-09T20:17:05.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.939+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=Q79CHOE8K9VU tag=HG9V3CGBN0GR type=write size=1048986 2024-05-09T20:17:05.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.943+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~QKK1UA6903CH.0 size=1048576 2024-05-09T20:17:05.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.947+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~QKK1UA6903CH.1 size=804771 2024-05-09T20:17:05.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.951+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=QKK1UA6903CH tag=OZPZHS6KUFXA type=write size=2901923 2024-05-09T20:17:05.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.951+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=9K4WISAYZFS1 type=del size=1580342 2024-05-09T20:17:05.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.955+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~M6MRIF1KUGTW.0 size=1048576 2024-05-09T20:17:05.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.955+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~M6MRIF1KUGTW.1 size=1048576 2024-05-09T20:17:05.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.959+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~M6MRIF1KUGTW.2 size=314072 2024-05-09T20:17:05.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.963+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=M6MRIF1KUGTW tag=VF83SFGJGI2N type=write size=3459800 2024-05-09T20:17:05.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.967+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~F0T9U55ASDHI.0 size=632072 2024-05-09T20:17:05.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.967+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=F0T9U55ASDHI tag=3584JO1JZU8J type=write size=1680648 2024-05-09T20:17:05.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.971+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~LWW2FWZ1KYFH.0 size=1048576 2024-05-09T20:17:05.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.971+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~LWW2FWZ1KYFH.1 size=906072 2024-05-09T20:17:05.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.979+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=LWW2FWZ1KYFH tag=JMLA430VTJGZ type=write size=3003224 2024-05-09T20:17:05.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.983+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=419JU72UMO39 type=del size=2246980 2024-05-09T20:17:05.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.987+0000 7fdc882e58c0 1 < suggested remove operation key=5H2GWM9LQELZ tag=6L9DUOIQX5XU type=del 2024-05-09T20:17:05.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.991+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=I2VYMIC4BVIJ type=del size=35097 2024-05-09T20:17:05.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.991+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=EZSOMX0J30FG type=del 2024-05-09T20:17:05.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:05.995+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=V139Q3CJTZK7 type=write size=923234 2024-05-09T20:17:05.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.003+0000 7fdc882e58c0 1 < suggested remove operation key=SNPNT5JFJOUS tag=21TIVMV2QY5V type=del 2024-05-09T20:17:05.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.003+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=03V1WBU4PZV0 type=write size=258598 2024-05-09T20:17:05.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.007+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~MXGE09BHXZ9M.0 size=1048576 2024-05-09T20:17:06.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.007+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~MXGE09BHXZ9M.1 size=1048576 2024-05-09T20:17:06.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.011+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~MXGE09BHXZ9M.2 size=469429 2024-05-09T20:17:06.008 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.015+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=MXGE09BHXZ9M tag=0N5DMPEJ527S type=write size=3615157 2024-05-09T20:17:06.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.019+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=M8R6WZW3692Z type=del size=3488064 2024-05-09T20:17:06.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.019+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=JHD5S8X1EO4A type=del 2024-05-09T20:17:06.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.027+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~SKAKT49FP13W.0 size=687629 2024-05-09T20:17:06.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.027+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=SKAKT49FP13W tag=VIS293HF376C type=write size=1736205 2024-05-09T20:17:06.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.027+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=5SGY03QU5JUE type=del 2024-05-09T20:17:06.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.035+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~WBSF1DVP21ZY.0 size=242404 2024-05-09T20:17:06.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.035+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=WBSF1DVP21ZY tag=9TWKIE4XQXD0 type=write size=1290980 2024-05-09T20:17:06.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.035+0000 7fdc882e58c0 1 < completed write operation key=EIMPFFK4I6NZ tag=GH6L20B6MRUK type=write size=442253 2024-05-09T20:17:06.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.039+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=Y4OVMK8QB0B6 type=write size=777098 2024-05-09T20:17:06.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.043+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=TJ66XP8VFF9P type=del 2024-05-09T20:17:06.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.047+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~BX1CPCRCQ5EO.0 size=469831 2024-05-09T20:17:06.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.047+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=BX1CPCRCQ5EO tag=E9MA9RBMO3KS type=write size=1518407 2024-05-09T20:17:06.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.051+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=6E6JWNX6F9QI type=del 2024-05-09T20:17:06.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.055+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=RHU4U8SGA2C1 type=write size=524228 2024-05-09T20:17:06.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.055+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~2VG7XJXOBL70.0 size=1048576 2024-05-09T20:17:06.052 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.059+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~2VG7XJXOBL70.1 size=1048576 2024-05-09T20:17:06.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.063+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~2VG7XJXOBL70.2 size=492586 2024-05-09T20:17:06.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.067+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=2VG7XJXOBL70 tag=VOE41A62U5SR type=write size=3638314 2024-05-09T20:17:06.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.071+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=7R1DZ7KWOXIC type=del size=1147279 2024-05-09T20:17:06.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.075+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~C313DHJD9LIT.0 size=182079 2024-05-09T20:17:06.073 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.079+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=C313DHJD9LIT tag=MQWBU9IA9TSW type=write size=1230655 2024-05-09T20:17:06.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.087+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~TEDITSMYJEF5.0 size=1048576 2024-05-09T20:17:06.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.091+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~TEDITSMYJEF5.1 size=1048576 2024-05-09T20:17:06.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.091+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~TEDITSMYJEF5.2 size=426579 2024-05-09T20:17:06.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.091+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=TEDITSMYJEF5 tag=CPD1NOLJN0E1 type=write size=3572307 2024-05-09T20:17:06.088 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.095+0000 7fdc882e58c0 1 < completed write operation key=EIMPFFK4I6NZ tag=YBLRC5Q6135Z type=write size=601564 2024-05-09T20:17:06.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.099+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=PZEOI6T0J8BI type=del size=2590036 2024-05-09T20:17:06.095 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.103+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~Q79CHOE8K9VU.0 size=410 2024-05-09T20:17:06.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.107+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=Q79CHOE8K9VU tag=HG9V3CGBN0GR type=write size=1048986 2024-05-09T20:17:06.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.111+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=UJWCFMFK6NQV type=del size=2073030 2024-05-09T20:17:06.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.115+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=9K4WISAYZFS1 type=del 2024-05-09T20:17:06.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.123+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~UQ2IAU0AVNI0.0 size=1048576 2024-05-09T20:17:06.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.123+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~UQ2IAU0AVNI0.1 size=899765 2024-05-09T20:17:06.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.123+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=UQ2IAU0AVNI0 tag=SFXJNZ0ZK0YC type=write size=2996917 2024-05-09T20:17:06.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.127+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~F0T9U55ASDHI.0 size=632072 2024-05-09T20:17:06.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.135+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=F0T9U55ASDHI tag=3584JO1JZU8J type=write size=1680648 2024-05-09T20:17:06.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.139+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=ZPAJL557E666 type=write size=982744 2024-05-09T20:17:06.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.143+0000 7fdc882e58c0 1 < suggested remove operation key=IZPPFLWVPRKO tag=419JU72UMO39 type=del 2024-05-09T20:17:06.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.147+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=I3CM7BILMINV type=del size=1708423 2024-05-09T20:17:06.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.151+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=I2VYMIC4BVIJ type=del 2024-05-09T20:17:06.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.155+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=TFVG38G51VOJ type=del size=2315909 2024-05-09T20:17:06.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.159+0000 7fdc882e58c0 1 < completed write operation key=BS0T1OQOBS67 tag=V139Q3CJTZK7 type=write size=923234 2024-05-09T20:17:06.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.163+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~B0N3RVPYQY2Z.0 size=1048576 2024-05-09T20:17:06.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.167+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~B0N3RVPYQY2Z.1 size=1048576 2024-05-09T20:17:06.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.171+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~B0N3RVPYQY2Z.2 size=851266 2024-05-09T20:17:06.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.171+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=B0N3RVPYQY2Z tag=X9WDK10JOXEL type=write size=3996994 2024-05-09T20:17:06.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.175+0000 7fdc882e58c0 1 < canceled operation key=346HV1XQKCOZ tag=03V1WBU4PZV0 type=write size=258598 2024-05-09T20:17:06.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.175+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=3OYA8R634HOA type=del size=1246161 2024-05-09T20:17:06.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.179+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=M8R6WZW3692Z type=del 2024-05-09T20:17:06.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.187+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~PMHGPG89PRQT.0 size=1048576 2024-05-09T20:17:06.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.187+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~PMHGPG89PRQT.1 size=893098 2024-05-09T20:17:06.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.187+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=PMHGPG89PRQT tag=NZ2EWV11XJCO type=write size=2990250 2024-05-09T20:17:06.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.191+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~SKAKT49FP13W.0 size=687629 2024-05-09T20:17:06.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.195+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=SKAKT49FP13W tag=VIS293HF376C type=write size=1736205 2024-05-09T20:17:06.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.199+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=9YLTZM50CTO7 type=del size=3196125 2024-05-09T20:17:06.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.203+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~WBSF1DVP21ZY.0 size=242404 2024-05-09T20:17:06.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.207+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=WBSF1DVP21ZY tag=9TWKIE4XQXD0 type=write size=1290980 2024-05-09T20:17:06.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.207+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=Z9FY94OETLE7 type=del size=217618 2024-05-09T20:17:06.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.211+0000 7fdc882e58c0 1 < completed write operation key=FNB9B18LIA8L tag=Y4OVMK8QB0B6 type=write size=777098 2024-05-09T20:17:06.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.215+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~29PCLPKIJR79.0 size=1048576 2024-05-09T20:17:06.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.219+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~29PCLPKIJR79.1 size=88321 2024-05-09T20:17:06.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.219+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=29PCLPKIJR79 tag=RZ04ROMYOYMJ type=write size=2185473 2024-05-09T20:17:06.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.219+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~BX1CPCRCQ5EO.0 size=469831 2024-05-09T20:17:06.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.223+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=BX1CPCRCQ5EO tag=E9MA9RBMO3KS type=write size=1518407 2024-05-09T20:17:06.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.231+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~GGF579VS68BK.0 size=788317 2024-05-09T20:17:06.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.231+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=GGF579VS68BK tag=MU5YYZ9R2MIO type=write size=1836893 2024-05-09T20:17:06.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.235+0000 7fdc882e58c0 1 < suggested remove operation key=AJANS5ADJO5B tag=RHU4U8SGA2C1 type=write 2024-05-09T20:17:06.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.239+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=X4RIOO7CYTC2 type=del size=2177037 2024-05-09T20:17:06.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.243+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=7R1DZ7KWOXIC type=del 2024-05-09T20:17:06.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.247+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~7W0LR9RZBDUJ.0 size=1048576 2024-05-09T20:17:06.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.251+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~7W0LR9RZBDUJ.1 size=1048576 2024-05-09T20:17:06.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.251+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~7W0LR9RZBDUJ.2 size=983714 2024-05-09T20:17:06.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.251+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=7W0LR9RZBDUJ tag=VO5I3KTL1DYA type=write size=4129442 2024-05-09T20:17:06.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.255+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~TEDITSMYJEF5.0 size=1048576 2024-05-09T20:17:06.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.259+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~TEDITSMYJEF5.1 size=1048576 2024-05-09T20:17:06.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.259+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~TEDITSMYJEF5.2 size=426579 2024-05-09T20:17:06.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.263+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=TEDITSMYJEF5 tag=CPD1NOLJN0E1 type=write size=3572307 2024-05-09T20:17:06.260 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.267+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=S1G56BMJS8LP type=del size=2677949 2024-05-09T20:17:06.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.271+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=PZEOI6T0J8BI type=del 2024-05-09T20:17:06.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.275+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=SR22DWQL9FZX type=write size=872249 2024-05-09T20:17:06.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.279+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=UJWCFMFK6NQV type=del 2024-05-09T20:17:06.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.279+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=BOM2GDG6JB38 type=del size=535866 2024-05-09T20:17:06.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.283+0000 7fdc882e58c0 1 < canceled part key=_multipart_SNPNT5JFJOUS.2~UQ2IAU0AVNI0.0 size=1048576 2024-05-09T20:17:06.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.283+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~UQ2IAU0AVNI0.1 size=899765 2024-05-09T20:17:06.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.287+0000 7fdc882e58c0 1 < canceled multipart upload key=SNPNT5JFJOUS upload=UQ2IAU0AVNI0 tag=SFXJNZ0ZK0YC type=write size=2996917 2024-05-09T20:17:06.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.291+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=WO92CQE1L4RM type=del size=1921917 2024-05-09T20:17:06.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.299+0000 7fdc882e58c0 1 < suggested remove operation key=CNV7NHP27JZ7 tag=ZPAJL557E666 type=write 2024-05-09T20:17:06.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.303+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~USU2BR9ZE5M9.0 size=1048576 2024-05-09T20:17:06.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.307+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~USU2BR9ZE5M9.1 size=1048576 2024-05-09T20:17:06.300 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.307+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~USU2BR9ZE5M9.2 size=31210 2024-05-09T20:17:06.300 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.307+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=USU2BR9ZE5M9 tag=IRTEHJRCYABC type=write size=3176938 2024-05-09T20:17:06.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.311+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=I3CM7BILMINV type=del 2024-05-09T20:17:06.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.315+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=BUH75XSS6PMP type=del size=3092632 2024-05-09T20:17:06.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.319+0000 7fdc882e58c0 1 < suggested remove operation key=HPK2T0629B1P tag=TFVG38G51VOJ type=del 2024-05-09T20:17:06.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.323+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=JOZ42335VKYG type=del size=1953053 2024-05-09T20:17:06.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.323+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~B0N3RVPYQY2Z.0 size=1048576 2024-05-09T20:17:06.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.327+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~B0N3RVPYQY2Z.1 size=1048576 2024-05-09T20:17:06.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.331+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~B0N3RVPYQY2Z.2 size=851266 2024-05-09T20:17:06.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.335+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=B0N3RVPYQY2Z tag=X9WDK10JOXEL type=write size=3996994 2024-05-09T20:17:06.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.339+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=XAMLXQI62DQO type=del size=1221948 2024-05-09T20:17:06.333 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.339+0000 7fdc882e58c0 -1 < failed to complete operation key=HPK2T0629B1P tag=3OYA8R634HOA type=del size=1246161: (22) Invalid argument 2024-05-09T20:17:06.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.343+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=02SBJ8H52CE9 type=del size=2644430 2024-05-09T20:17:06.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.347+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~PMHGPG89PRQT.0 size=1048576 2024-05-09T20:17:06.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.347+0000 7fdc882e58c0 1 < canceled part key=_multipart_TQ5ZHATLNVPP.2~PMHGPG89PRQT.1 size=893098 2024-05-09T20:17:06.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.351+0000 7fdc882e58c0 1 < canceled multipart upload key=TQ5ZHATLNVPP upload=PMHGPG89PRQT tag=NZ2EWV11XJCO type=write size=2990250 2024-05-09T20:17:06.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.355+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=DZO21H33GG6N type=del size=3145358 2024-05-09T20:17:06.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.359+0000 7fdc882e58c0 1 < suggested update operation key=IZPPFLWVPRKO tag=9YLTZM50CTO7 type=del size=3196125 2024-05-09T20:17:06.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.363+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=DCV1GB11YP8L type=write size=347666 2024-05-09T20:17:06.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.363+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=Z9FY94OETLE7 type=del 2024-05-09T20:17:06.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.371+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~EU2RXUF159G0.0 size=1048576 2024-05-09T20:17:06.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.371+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~EU2RXUF159G0.1 size=1048576 2024-05-09T20:17:06.367 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.375+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~EU2RXUF159G0.2 size=952780 2024-05-09T20:17:06.367 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.375+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=EU2RXUF159G0 tag=CRG2IKCAMSZ7 type=write size=4098508 2024-05-09T20:17:06.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.375+0000 7fdc882e58c0 1 < canceled part key=_multipart_NYHR6GTPQYZS.2~29PCLPKIJR79.0 size=1048576 2024-05-09T20:17:06.370 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.379+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~29PCLPKIJR79.1 size=88321 2024-05-09T20:17:06.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.383+0000 7fdc882e58c0 1 < canceled multipart upload key=NYHR6GTPQYZS upload=29PCLPKIJR79 tag=RZ04ROMYOYMJ type=write size=2185473 2024-05-09T20:17:06.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.387+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~3P31XRFVZ37C.0 size=1048576 2024-05-09T20:17:06.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.391+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~3P31XRFVZ37C.1 size=1048576 2024-05-09T20:17:06.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.395+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~3P31XRFVZ37C.2 size=92823 2024-05-09T20:17:06.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.395+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=3P31XRFVZ37C tag=PGPAMUI6DFW6 type=write size=3238551 2024-05-09T20:17:06.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.395+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~GGF579VS68BK.0 size=788317 2024-05-09T20:17:06.393 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.399+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=GGF579VS68BK tag=MU5YYZ9R2MIO type=write size=1836893 2024-05-09T20:17:06.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.403+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=3XDC2KSN6963 type=del size=256946 2024-05-09T20:17:06.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.407+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=X4RIOO7CYTC2 type=del 2024-05-09T20:17:06.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.411+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=395LQHMR41BA type=del size=2953378 2024-05-09T20:17:06.405 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.411+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~7W0LR9RZBDUJ.0 size=1048576 2024-05-09T20:17:06.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.415+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~7W0LR9RZBDUJ.1 size=1048576 2024-05-09T20:17:06.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.415+0000 7fdc882e58c0 1 < canceled part key=_multipart_T6T3IOW9VPA0.2~7W0LR9RZBDUJ.2 size=983714 2024-05-09T20:17:06.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.419+0000 7fdc882e58c0 1 < canceled multipart upload key=T6T3IOW9VPA0 upload=7W0LR9RZBDUJ tag=VO5I3KTL1DYA type=write size=4129442 2024-05-09T20:17:06.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.423+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=73O0X0YI2FWX type=del size=3794714 2024-05-09T20:17:06.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.427+0000 7fdc882e58c0 1 < suggested update operation key=T6T3IOW9VPA0 tag=S1G56BMJS8LP type=del size=2677949 2024-05-09T20:17:06.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.431+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=A0IR34FDPLGI type=del size=232186 2024-05-09T20:17:06.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.435+0000 7fdc882e58c0 1 < completed write operation key=SPXHCWZICCWY tag=SR22DWQL9FZX type=write size=872249 2024-05-09T20:17:06.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.439+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=V2KZUS55WXOY type=del size=3942297 2024-05-09T20:17:06.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.443+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=BOM2GDG6JB38 type=del 2024-05-09T20:17:06.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.447+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~SV73B7SDCLN8.0 size=1048576 2024-05-09T20:17:06.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.451+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~SV73B7SDCLN8.1 size=1048576 2024-05-09T20:17:06.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.455+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~SV73B7SDCLN8.2 size=938873 2024-05-09T20:17:06.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.455+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=SV73B7SDCLN8 tag=5MUSGUV1R0H4 type=write size=4084601 2024-05-09T20:17:06.450 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.459+0000 7fdc882e58c0 -1 < failed to complete operation key=HPK2T0629B1P tag=WO92CQE1L4RM type=del size=1921917: (22) Invalid argument 2024-05-09T20:17:06.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.463+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=P2AB7KJ1WZ95 type=del size=1330112 2024-05-09T20:17:06.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.463+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~USU2BR9ZE5M9.0 size=1048576 2024-05-09T20:17:06.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.467+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~USU2BR9ZE5M9.1 size=1048576 2024-05-09T20:17:06.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.471+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~USU2BR9ZE5M9.2 size=31210 2024-05-09T20:17:06.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.475+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=USU2BR9ZE5M9 tag=IRTEHJRCYABC type=write size=3176938 2024-05-09T20:17:06.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.483+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~CXZOX5BYOYX6.0 size=331330 2024-05-09T20:17:06.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.483+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=CXZOX5BYOYX6 tag=P0IBVN2VCQXZ type=write size=1379906 2024-05-09T20:17:06.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.487+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=BUH75XSS6PMP type=del 2024-05-09T20:17:06.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.487+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=HC11BEWUYILG type=del size=2006449 2024-05-09T20:17:06.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.491+0000 7fdc882e58c0 -1 < failed to complete operation key=T6T3IOW9VPA0 tag=JOZ42335VKYG type=del size=1953053: (22) Invalid argument 2024-05-09T20:17:06.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.499+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~5ASS5L1J3M50.0 size=549231 2024-05-09T20:17:06.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.499+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=5ASS5L1J3M50 tag=TCHW3PC9R7CF type=write size=1597807 2024-05-09T20:17:06.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.499+0000 7fdc882e58c0 1 < canceled operation key=CNV7NHP27JZ7 tag=XAMLXQI62DQO type=del size=1221948 2024-05-09T20:17:06.497 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.503+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=5AFNHYUDTQRR type=del size=2069175 2024-05-09T20:17:06.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.511+0000 7fdc882e58c0 1 < suggested update operation key=G0DX1F257RLD tag=02SBJ8H52CE9 type=del size=2644430 2024-05-09T20:17:06.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.515+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=7WF376Q92QAI type=del size=2264323 2024-05-09T20:17:06.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.519+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=DZO21H33GG6N type=del 2024-05-09T20:17:06.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.523+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~WTC03RN0E5HR.0 size=1048576 2024-05-09T20:17:06.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.527+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~WTC03RN0E5HR.1 size=1048576 2024-05-09T20:17:06.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.527+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~WTC03RN0E5HR.2 size=856109 2024-05-09T20:17:06.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.527+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=WTC03RN0E5HR tag=ZQ5E7ANK9PLT type=write size=4001837 2024-05-09T20:17:06.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.531+0000 7fdc882e58c0 1 < completed write operation key=VZ7XOAT899GU tag=DCV1GB11YP8L type=write size=347666 2024-05-09T20:17:06.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.535+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=FTMLXTS2JS4X type=del size=768888 2024-05-09T20:17:06.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.539+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~EU2RXUF159G0.0 size=1048576 2024-05-09T20:17:06.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.539+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~EU2RXUF159G0.1 size=1048576 2024-05-09T20:17:06.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.543+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~EU2RXUF159G0.2 size=952780 2024-05-09T20:17:06.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.547+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=EU2RXUF159G0 tag=CRG2IKCAMSZ7 type=write size=4098508 2024-05-09T20:17:06.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.555+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=TX3AAZSRBWTF type=del size=816507 2024-05-09T20:17:06.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.559+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~3P31XRFVZ37C.0 size=1048576 2024-05-09T20:17:06.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.559+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~3P31XRFVZ37C.1 size=1048576 2024-05-09T20:17:06.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.563+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~3P31XRFVZ37C.2 size=92823 2024-05-09T20:17:06.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.567+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=3P31XRFVZ37C tag=PGPAMUI6DFW6 type=write size=3238551 2024-05-09T20:17:06.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.571+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=8YZL0TIFOZVQ type=del size=883247 2024-05-09T20:17:06.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.575+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=3XDC2KSN6963 type=del 2024-05-09T20:17:06.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.579+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~C5UOQRFJEMQO.0 size=1048576 2024-05-09T20:17:06.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.583+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~C5UOQRFJEMQO.1 size=1048576 2024-05-09T20:17:06.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.587+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~C5UOQRFJEMQO.2 size=857398 2024-05-09T20:17:06.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.587+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=C5UOQRFJEMQO tag=2U8UMB9ARTA1 type=write size=4003126 2024-05-09T20:17:06.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.587+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=395LQHMR41BA type=del 2024-05-09T20:17:06.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.591+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=84BCZAWV4HUY type=del size=2099779 2024-05-09T20:17:06.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.595+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=73O0X0YI2FWX type=del 2024-05-09T20:17:06.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.599+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=L58OOIKB1VJY type=del size=2514550 2024-05-09T20:17:06.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.603+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=A0IR34FDPLGI type=del 2024-05-09T20:17:06.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.607+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~7BS9I0DM1R51.0 size=1048576 2024-05-09T20:17:06.603 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.611+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~7BS9I0DM1R51.1 size=1048576 2024-05-09T20:17:06.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.615+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~7BS9I0DM1R51.2 size=266756 2024-05-09T20:17:06.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.615+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=7BS9I0DM1R51 tag=CZE6LFGVVES9 type=write size=3412484 2024-05-09T20:17:06.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.615+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=V2KZUS55WXOY type=del 2024-05-09T20:17:06.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.623+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~RLC1XM9FL2CD.0 size=1048576 2024-05-09T20:17:06.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.627+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~RLC1XM9FL2CD.1 size=353257 2024-05-09T20:17:06.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.627+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=RLC1XM9FL2CD tag=P1KYIIPIJXCS type=write size=2450409 2024-05-09T20:17:06.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.631+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~SV73B7SDCLN8.0 size=1048576 2024-05-09T20:17:06.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.631+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~SV73B7SDCLN8.1 size=1048576 2024-05-09T20:17:06.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.635+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~SV73B7SDCLN8.2 size=938873 2024-05-09T20:17:06.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.643+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=SV73B7SDCLN8 tag=5MUSGUV1R0H4 type=write size=4084601 2024-05-09T20:17:06.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.647+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~GKR6VRRCR5Z2.0 size=1048576 2024-05-09T20:17:06.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.651+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~GKR6VRRCR5Z2.1 size=574647 2024-05-09T20:17:06.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.651+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=GKR6VRRCR5Z2 tag=2AIX9X3ZO8YD type=write size=2671799 2024-05-09T20:17:06.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.659+0000 7fdc882e58c0 1 < suggested remove operation key=NYHR6GTPQYZS tag=P2AB7KJ1WZ95 type=del 2024-05-09T20:17:06.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.659+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=SL1RULAWL9S1 type=write size=320564 2024-05-09T20:17:06.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.663+0000 7fdc882e58c0 1 < canceled part key=_multipart_AJANS5ADJO5B.2~CXZOX5BYOYX6.0 size=331330 2024-05-09T20:17:06.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.667+0000 7fdc882e58c0 1 < canceled multipart upload key=AJANS5ADJO5B upload=CXZOX5BYOYX6 tag=P0IBVN2VCQXZ type=write size=1379906 2024-05-09T20:17:06.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.675+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~FMRSPCBPHMUQ.0 size=1048576 2024-05-09T20:17:06.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.675+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~FMRSPCBPHMUQ.1 size=1048576 2024-05-09T20:17:06.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.679+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~FMRSPCBPHMUQ.2 size=858032 2024-05-09T20:17:06.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.679+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=FMRSPCBPHMUQ tag=PCR3VLNBY19P type=write size=4003760 2024-05-09T20:17:06.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.687+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=HC11BEWUYILG type=del size=2006449 2024-05-09T20:17:06.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.691+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=Q9XSF56PF74M type=del size=3979977 2024-05-09T20:17:06.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.691+0000 7fdc882e58c0 1 < canceled part key=_multipart_JS90W6PIREMD.2~5ASS5L1J3M50.0 size=549231 2024-05-09T20:17:06.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.695+0000 7fdc882e58c0 1 < canceled multipart upload key=JS90W6PIREMD upload=5ASS5L1J3M50 tag=TCHW3PC9R7CF type=write size=1597807 2024-05-09T20:17:06.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.703+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~IDBR0LXIC3Z0.0 size=1048576 2024-05-09T20:17:06.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.703+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~IDBR0LXIC3Z0.1 size=424621 2024-05-09T20:17:06.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.703+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=IDBR0LXIC3Z0 tag=NRN1DSOU2ZOM type=write size=2521773 2024-05-09T20:17:06.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.711+0000 7fdc882e58c0 1 < suggested remove operation key=AJANS5ADJO5B tag=5AFNHYUDTQRR type=del 2024-05-09T20:17:06.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.715+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=T6U0YG2UABZN type=write size=53085 2024-05-09T20:17:06.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.719+0000 7fdc882e58c0 1 < suggested remove operation key=G0DX1F257RLD tag=7WF376Q92QAI type=del 2024-05-09T20:17:06.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.723+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=LPF6XU308R06 type=del size=686819 2024-05-09T20:17:06.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.727+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~WTC03RN0E5HR.0 size=1048576 2024-05-09T20:17:06.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.731+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~WTC03RN0E5HR.1 size=1048576 2024-05-09T20:17:06.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.735+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~WTC03RN0E5HR.2 size=856109 2024-05-09T20:17:06.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.739+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=WTC03RN0E5HR tag=ZQ5E7ANK9PLT type=write size=4001837 2024-05-09T20:17:06.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.743+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=Y6XV70ALH9BJ type=del size=639825 2024-05-09T20:17:06.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.747+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=FTMLXTS2JS4X type=del 2024-05-09T20:17:06.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.751+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~84X78JTK4QME.0 size=1048576 2024-05-09T20:17:06.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.755+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~84X78JTK4QME.1 size=442413 2024-05-09T20:17:06.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.755+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=84X78JTK4QME tag=RQWKDJIFB7E3 type=write size=2539565 2024-05-09T20:17:06.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.759+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=TX3AAZSRBWTF type=del 2024-05-09T20:17:06.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.763+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~Q1UECD24VV46.0 size=859873 2024-05-09T20:17:06.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.763+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=Q1UECD24VV46 tag=6RJK4JZ3S3YR type=write size=1908449 2024-05-09T20:17:06.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.767+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=8YZL0TIFOZVQ type=del 2024-05-09T20:17:06.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.771+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~TKRD4MLL1EER.0 size=613659 2024-05-09T20:17:06.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.771+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=TKRD4MLL1EER tag=UBJMDG1TUK39 type=write size=1662235 2024-05-09T20:17:06.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.775+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~C5UOQRFJEMQO.0 size=1048576 2024-05-09T20:17:06.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.779+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~C5UOQRFJEMQO.1 size=1048576 2024-05-09T20:17:06.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.783+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~C5UOQRFJEMQO.2 size=857398 2024-05-09T20:17:06.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.787+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=C5UOQRFJEMQO tag=2U8UMB9ARTA1 type=write size=4003126 2024-05-09T20:17:06.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.791+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~PPMWW0UM2Z3K.0 size=472745 2024-05-09T20:17:06.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.791+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=PPMWW0UM2Z3K tag=OVK310OOHSPW type=write size=1521321 2024-05-09T20:17:06.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.799+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=84BCZAWV4HUY type=del size=2099779 2024-05-09T20:17:06.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.803+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=61OJXBTT5JIP type=del size=335939 2024-05-09T20:17:06.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.807+0000 7fdc882e58c0 -1 < failed to complete operation key=TQ5ZHATLNVPP tag=L58OOIKB1VJY type=del size=2514550: (22) Invalid argument 2024-05-09T20:17:06.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.811+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~ZSL2W1E0XHXN.0 size=1048576 2024-05-09T20:17:06.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.815+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~ZSL2W1E0XHXN.1 size=652437 2024-05-09T20:17:06.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.815+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=ZSL2W1E0XHXN tag=4UOZFWQJ4QQ0 type=write size=2749589 2024-05-09T20:17:06.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.819+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~7BS9I0DM1R51.0 size=1048576 2024-05-09T20:17:06.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.819+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~7BS9I0DM1R51.1 size=1048576 2024-05-09T20:17:06.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.823+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~7BS9I0DM1R51.2 size=266756 2024-05-09T20:17:06.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.831+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=7BS9I0DM1R51 tag=CZE6LFGVVES9 type=write size=3412484 2024-05-09T20:17:06.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.831+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=2NKIF1QW1HET type=del size=3218488 2024-05-09T20:17:06.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.835+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~RLC1XM9FL2CD.0 size=1048576 2024-05-09T20:17:06.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.839+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~RLC1XM9FL2CD.1 size=353257 2024-05-09T20:17:06.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.843+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=RLC1XM9FL2CD tag=P1KYIIPIJXCS type=write size=2450409 2024-05-09T20:17:06.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.847+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=IJTX0HCY86D5 type=del size=3119011 2024-05-09T20:17:06.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.851+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~GKR6VRRCR5Z2.0 size=1048576 2024-05-09T20:17:06.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.855+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~GKR6VRRCR5Z2.1 size=574647 2024-05-09T20:17:06.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.863+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=GKR6VRRCR5Z2 tag=2AIX9X3ZO8YD type=write size=2671799 2024-05-09T20:17:06.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.867+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=0JCMEBMFW0HT type=write size=625421 2024-05-09T20:17:06.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.875+0000 7fdc882e58c0 1 < suggested remove operation key=TQ5ZHATLNVPP tag=SL1RULAWL9S1 type=write 2024-05-09T20:17:06.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.875+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=KOIJGJP7986W type=del size=3312972 2024-05-09T20:17:06.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.879+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~FMRSPCBPHMUQ.0 size=1048576 2024-05-09T20:17:06.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.879+0000 7fdc882e58c0 1 < canceled part key=_multipart_8FFZZDURUWM1.2~FMRSPCBPHMUQ.1 size=1048576 2024-05-09T20:17:06.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.883+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~FMRSPCBPHMUQ.2 size=858032 2024-05-09T20:17:06.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.887+0000 7fdc882e58c0 1 < canceled multipart upload key=8FFZZDURUWM1 upload=FMRSPCBPHMUQ tag=PCR3VLNBY19P type=write size=4003760 2024-05-09T20:17:06.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.891+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=0X1QPBZVQQIK type=del size=1671150 2024-05-09T20:17:06.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.895+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=Q9XSF56PF74M type=del 2024-05-09T20:17:06.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.903+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~SXTUVW4VQVYB.0 size=1048576 2024-05-09T20:17:06.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.903+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~SXTUVW4VQVYB.1 size=1048576 2024-05-09T20:17:06.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.907+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~SXTUVW4VQVYB.2 size=714023 2024-05-09T20:17:06.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.907+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=SXTUVW4VQVYB tag=V9U3P2TW70KM type=write size=3859751 2024-05-09T20:17:06.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.911+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~IDBR0LXIC3Z0.0 size=1048576 2024-05-09T20:17:06.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.915+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~IDBR0LXIC3Z0.1 size=424621 2024-05-09T20:17:06.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.919+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=IDBR0LXIC3Z0 tag=NRN1DSOU2ZOM type=write size=2521773 2024-05-09T20:17:06.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.927+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~U34FY5M6AD17.0 size=157271 2024-05-09T20:17:06.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.927+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=U34FY5M6AD17 tag=RVS1LWA25RTV type=write size=1205847 2024-05-09T20:17:06.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.927+0000 7fdc882e58c0 1 < completed write operation key=HPK2T0629B1P tag=T6U0YG2UABZN type=write size=53085 2024-05-09T20:17:06.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.931+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=3NX2WBHNYHM0 type=del size=3881511 2024-05-09T20:17:06.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.939+0000 7fdc882e58c0 1 < suggested remove operation key=O07UHAIJZF0R tag=LPF6XU308R06 type=del 2024-05-09T20:17:06.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.943+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=5J71VFQ1OEG6 type=del size=3371388 2024-05-09T20:17:06.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.947+0000 7fdc882e58c0 1 < suggested update operation key=T5EL7X0P450L tag=Y6XV70ALH9BJ type=del size=639825 2024-05-09T20:17:06.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.951+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=AZMJMEFB92A4 type=del size=1108356 2024-05-09T20:17:06.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.955+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~84X78JTK4QME.0 size=1048576 2024-05-09T20:17:06.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.959+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~84X78JTK4QME.1 size=442413 2024-05-09T20:17:06.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.963+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=84X78JTK4QME tag=RQWKDJIFB7E3 type=write size=2539565 2024-05-09T20:17:06.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.971+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~ELFZ29DX89ZX.0 size=1048576 2024-05-09T20:17:06.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.975+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~ELFZ29DX89ZX.1 size=1048576 2024-05-09T20:17:06.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.975+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~ELFZ29DX89ZX.2 size=942094 2024-05-09T20:17:06.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.975+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=ELFZ29DX89ZX tag=A3S61PAITUD2 type=write size=4087822 2024-05-09T20:17:06.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.979+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~Q1UECD24VV46.0 size=859873 2024-05-09T20:17:06.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.987+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=Q1UECD24VV46 tag=6RJK4JZ3S3YR type=write size=1908449 2024-05-09T20:17:06.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.991+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~JAZOVNBJICT6.0 size=1048576 2024-05-09T20:17:06.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.995+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~JAZOVNBJICT6.1 size=1048576 2024-05-09T20:17:06.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.999+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~JAZOVNBJICT6.2 size=574187 2024-05-09T20:17:06.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:06.999+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=JAZOVNBJICT6 tag=A2P24AHMCGSI type=write size=3719915 2024-05-09T20:17:06.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.003+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~TKRD4MLL1EER.0 size=613659 2024-05-09T20:17:06.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.007+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=TKRD4MLL1EER tag=UBJMDG1TUK39 type=write size=1662235 2024-05-09T20:17:07.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.011+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=A6ZMF7AQTY4I type=del size=2899993 2024-05-09T20:17:07.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.015+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~PPMWW0UM2Z3K.0 size=472745 2024-05-09T20:17:07.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.019+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=PPMWW0UM2Z3K tag=OVK310OOHSPW type=write size=1521321 2024-05-09T20:17:07.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.027+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~3FMN18RQSLYB.0 size=1048576 2024-05-09T20:17:07.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.031+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~3FMN18RQSLYB.1 size=1048576 2024-05-09T20:17:07.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.031+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~3FMN18RQSLYB.2 size=433469 2024-05-09T20:17:07.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.031+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=3FMN18RQSLYB tag=MO74TD87N9YG type=write size=3579197 2024-05-09T20:17:07.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.039+0000 7fdc882e58c0 1 < suggested update operation key=5NEOR8OOFAFZ tag=61OJXBTT5JIP type=del size=335939 2024-05-09T20:17:07.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.043+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=HS53IZMSIIL5 type=write size=3289 2024-05-09T20:17:07.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.047+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~ZSL2W1E0XHXN.0 size=1048576 2024-05-09T20:17:07.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.051+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~ZSL2W1E0XHXN.1 size=652437 2024-05-09T20:17:07.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.055+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=ZSL2W1E0XHXN tag=4UOZFWQJ4QQ0 type=write size=2749589 2024-05-09T20:17:07.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.063+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~9X4W39LSL44H.0 size=640461 2024-05-09T20:17:07.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.063+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=9X4W39LSL44H tag=UFTQ61E3WZ1J type=write size=1689037 2024-05-09T20:17:07.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.067+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=2NKIF1QW1HET type=del 2024-05-09T20:17:07.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.067+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=01I36BT3ULUW type=del size=1193034 2024-05-09T20:17:07.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.075+0000 7fdc882e58c0 1 < suggested remove operation key=SPXHCWZICCWY tag=IJTX0HCY86D5 type=del 2024-05-09T20:17:07.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.079+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=76SC493CL1R7 type=del size=53711 2024-05-09T20:17:07.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.083+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=0JCMEBMFW0HT type=write size=625421 2024-05-09T20:17:07.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.087+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=E44C354XF9DL type=del size=291052 2024-05-09T20:17:07.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.087+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=KOIJGJP7986W type=del 2024-05-09T20:17:07.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.095+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~H0Q6UDHP1AES.0 size=397295 2024-05-09T20:17:07.088 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.095+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=H0Q6UDHP1AES tag=O6YJIECO7P0S type=write size=1445871 2024-05-09T20:17:07.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.099+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=0X1QPBZVQQIK type=del 2024-05-09T20:17:07.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.099+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=FH5GXDK27ZXL type=del size=2571838 2024-05-09T20:17:07.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.103+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~SXTUVW4VQVYB.0 size=1048576 2024-05-09T20:17:07.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.107+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~SXTUVW4VQVYB.1 size=1048576 2024-05-09T20:17:07.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.111+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~SXTUVW4VQVYB.2 size=714023 2024-05-09T20:17:07.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.115+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=SXTUVW4VQVYB tag=V9U3P2TW70KM type=write size=3859751 2024-05-09T20:17:07.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.119+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~ABII8LMVPUQI.0 size=1048576 2024-05-09T20:17:07.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.123+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~ABII8LMVPUQI.1 size=306478 2024-05-09T20:17:07.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.123+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=ABII8LMVPUQI tag=WZ46J198P39E type=write size=2403630 2024-05-09T20:17:07.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.127+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~U34FY5M6AD17.0 size=157271 2024-05-09T20:17:07.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.131+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=U34FY5M6AD17 tag=RVS1LWA25RTV type=write size=1205847 2024-05-09T20:17:07.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.139+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~TCRPO44YW3BX.0 size=289705 2024-05-09T20:17:07.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.139+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=TCRPO44YW3BX tag=5WBCSRH685X3 type=write size=1338281 2024-05-09T20:17:07.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.143+0000 7fdc882e58c0 1 < suggested update operation key=TWWOL5D8C1MQ tag=3NX2WBHNYHM0 type=del size=3881511 2024-05-09T20:17:07.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.151+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~X6T8TJ4CCXTG.0 size=75447 2024-05-09T20:17:07.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.151+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=X6T8TJ4CCXTG tag=L2KITJZCP3NS type=write size=1124023 2024-05-09T20:17:07.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.159+0000 7fdc882e58c0 1 < suggested remove operation key=IZPPFLWVPRKO tag=5J71VFQ1OEG6 type=del 2024-05-09T20:17:07.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.163+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=22ZNRDWV7OHK type=del size=3883799 2024-05-09T20:17:07.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.167+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=AZMJMEFB92A4 type=del 2024-05-09T20:17:07.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.171+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=MD6F7YOZ8YUK type=del size=3454986 2024-05-09T20:17:07.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.171+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~ELFZ29DX89ZX.0 size=1048576 2024-05-09T20:17:07.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.175+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~ELFZ29DX89ZX.1 size=1048576 2024-05-09T20:17:07.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.183+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~ELFZ29DX89ZX.2 size=942094 2024-05-09T20:17:07.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.187+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=ELFZ29DX89ZX tag=A3S61PAITUD2 type=write size=4087822 2024-05-09T20:17:07.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.195+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~ITMP7P9N0MXP.0 size=1048576 2024-05-09T20:17:07.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.199+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~ITMP7P9N0MXP.1 size=1048576 2024-05-09T20:17:07.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.199+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~ITMP7P9N0MXP.2 size=928531 2024-05-09T20:17:07.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.199+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=ITMP7P9N0MXP tag=PTSLO1ZOLHKR type=write size=4074259 2024-05-09T20:17:07.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.199+0000 7fdc882e58c0 1 < canceled part key=_multipart_CNV7NHP27JZ7.2~JAZOVNBJICT6.0 size=1048576 2024-05-09T20:17:07.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.199+0000 7fdc882e58c0 1 < canceled part key=_multipart_CNV7NHP27JZ7.2~JAZOVNBJICT6.1 size=1048576 2024-05-09T20:17:07.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.203+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~JAZOVNBJICT6.2 size=574187 2024-05-09T20:17:07.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.211+0000 7fdc882e58c0 1 < canceled multipart upload key=CNV7NHP27JZ7 upload=JAZOVNBJICT6 tag=A2P24AHMCGSI type=write size=3719915 2024-05-09T20:17:07.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.219+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~WKXAO7KH4NEI.0 size=1048576 2024-05-09T20:17:07.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.219+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~WKXAO7KH4NEI.1 size=1046167 2024-05-09T20:17:07.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.219+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=WKXAO7KH4NEI tag=ZA9RCALRM2DE type=write size=3143319 2024-05-09T20:17:07.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.223+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=A6ZMF7AQTY4I type=del 2024-05-09T20:17:07.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.231+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=V4BMACR51VVR type=del size=1824921 2024-05-09T20:17:07.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.235+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~3FMN18RQSLYB.0 size=1048576 2024-05-09T20:17:07.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.235+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~3FMN18RQSLYB.1 size=1048576 2024-05-09T20:17:07.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.239+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~3FMN18RQSLYB.2 size=433469 2024-05-09T20:17:07.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.247+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=3FMN18RQSLYB tag=MO74TD87N9YG type=write size=3579197 2024-05-09T20:17:07.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.255+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~T4HJQL9PNATS.0 size=240803 2024-05-09T20:17:07.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.255+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=T4HJQL9PNATS tag=68HWJ9YUK7TC type=write size=1289379 2024-05-09T20:17:07.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.259+0000 7fdc882e58c0 -1 < failed to complete operation key=SPXHCWZICCWY tag=HS53IZMSIIL5 type=write size=3289: (22) Invalid argument 2024-05-09T20:17:07.254 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.263+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=JFNNXOEFHEK7 type=del size=3663730 2024-05-09T20:17:07.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.267+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~9X4W39LSL44H.0 size=640461 2024-05-09T20:17:07.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.271+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=9X4W39LSL44H tag=UFTQ61E3WZ1J type=write size=1689037 2024-05-09T20:17:07.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.275+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=OYE3HVMUZKU3 type=write size=39977 2024-05-09T20:17:07.272 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.279+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=01I36BT3ULUW type=del 2024-05-09T20:17:07.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.283+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=NGBVHPCZK8ZF type=del size=219628 2024-05-09T20:17:07.280 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.287+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=76SC493CL1R7 type=del 2024-05-09T20:17:07.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.291+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=ZCFDEVNFS8VI type=del size=584009 2024-05-09T20:17:07.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.295+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=E44C354XF9DL type=del 2024-05-09T20:17:07.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.299+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=15V3H2E6W9N5 type=del size=94844 2024-05-09T20:17:07.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.303+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~H0Q6UDHP1AES.0 size=397295 2024-05-09T20:17:07.299 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.307+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=H0Q6UDHP1AES tag=O6YJIECO7P0S type=write size=1445871 2024-05-09T20:17:07.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.311+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=O04X15MTTEH5 type=del size=1125095 2024-05-09T20:17:07.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.315+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=FH5GXDK27ZXL type=del 2024-05-09T20:17:07.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.319+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~D5HLYOIL1ETL.0 size=911598 2024-05-09T20:17:07.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.319+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=D5HLYOIL1ETL tag=U0XRVIKZ2E4B type=write size=1960174 2024-05-09T20:17:07.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.323+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~ABII8LMVPUQI.0 size=1048576 2024-05-09T20:17:07.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.327+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~ABII8LMVPUQI.1 size=306478 2024-05-09T20:17:07.325 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.331+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=ABII8LMVPUQI tag=WZ46J198P39E type=write size=2403630 2024-05-09T20:17:07.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.335+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=KI6W1C9KNWJ6 type=del size=27064 2024-05-09T20:17:07.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.339+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~TCRPO44YW3BX.0 size=289705 2024-05-09T20:17:07.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.347+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=TCRPO44YW3BX tag=5WBCSRH685X3 type=write size=1338281 2024-05-09T20:17:07.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.351+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~5JQ70JYXJW3I.0 size=1048576 2024-05-09T20:17:07.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.355+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~5JQ70JYXJW3I.1 size=1048576 2024-05-09T20:17:07.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.359+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~5JQ70JYXJW3I.2 size=577090 2024-05-09T20:17:07.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.359+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=5JQ70JYXJW3I tag=HAF758W6QRSJ type=write size=3722818 2024-05-09T20:17:07.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.359+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~X6T8TJ4CCXTG.0 size=75447 2024-05-09T20:17:07.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.367+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=X6T8TJ4CCXTG tag=L2KITJZCP3NS type=write size=1124023 2024-05-09T20:17:07.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.371+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=P1U1W99G2LK8 type=del size=2099742 2024-05-09T20:17:07.367 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.375+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=22ZNRDWV7OHK type=del 2024-05-09T20:17:07.370 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.379+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=IJQ2OHEO9SXG type=del size=3404024 2024-05-09T20:17:07.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.379+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=MD6F7YOZ8YUK type=del 2024-05-09T20:17:07.377 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.383+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=I1CAR18YL1EX type=del size=2444958 2024-05-09T20:17:07.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.387+0000 7fdc882e58c0 1 < canceled part key=_multipart_SPXHCWZICCWY.2~ITMP7P9N0MXP.0 size=1048576 2024-05-09T20:17:07.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.387+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~ITMP7P9N0MXP.1 size=1048576 2024-05-09T20:17:07.384 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.391+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~ITMP7P9N0MXP.2 size=928531 2024-05-09T20:17:07.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.395+0000 7fdc882e58c0 1 < canceled multipart upload key=SPXHCWZICCWY upload=ITMP7P9N0MXP tag=PTSLO1ZOLHKR type=write size=4074259 2024-05-09T20:17:07.393 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.399+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=D93V1D6W6LTY type=del size=1379197 2024-05-09T20:17:07.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.403+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~WKXAO7KH4NEI.0 size=1048576 2024-05-09T20:17:07.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.407+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~WKXAO7KH4NEI.1 size=1046167 2024-05-09T20:17:07.405 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.411+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=WKXAO7KH4NEI tag=ZA9RCALRM2DE type=write size=3143319 2024-05-09T20:17:07.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.415+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=56FIBUSQ0ZQ4 type=del size=2440405 2024-05-09T20:17:07.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.419+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=V4BMACR51VVR type=del 2024-05-09T20:17:07.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.423+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~92QGYOSLK7UX.0 size=138332 2024-05-09T20:17:07.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.423+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=92QGYOSLK7UX tag=O08SSXQC5WRY type=write size=1186908 2024-05-09T20:17:07.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.427+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~T4HJQL9PNATS.0 size=240803 2024-05-09T20:17:07.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.435+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=T4HJQL9PNATS tag=68HWJ9YUK7TC type=write size=1289379 2024-05-09T20:17:07.430 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.439+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=9ZJ63RHYDUAA type=del size=1396434 2024-05-09T20:17:07.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.439+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=JFNNXOEFHEK7 type=del 2024-05-09T20:17:07.440 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.447+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~FWO7YZSOA38E.0 size=1048576 2024-05-09T20:17:07.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.451+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~FWO7YZSOA38E.1 size=246827 2024-05-09T20:17:07.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.451+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=FWO7YZSOA38E tag=LGB1C04VJBGP type=write size=2343979 2024-05-09T20:17:07.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.455+0000 7fdc882e58c0 1 < suggested remove operation key=346HV1XQKCOZ tag=OYE3HVMUZKU3 type=write 2024-05-09T20:17:07.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.463+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~LYLVLYCPYV0W.0 size=1048576 2024-05-09T20:17:07.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.463+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~LYLVLYCPYV0W.1 size=1048576 2024-05-09T20:17:07.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.467+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~LYLVLYCPYV0W.2 size=924888 2024-05-09T20:17:07.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.467+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=LYLVLYCPYV0W tag=69SFOR6GD2JV type=write size=4070616 2024-05-09T20:17:07.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.471+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=NGBVHPCZK8ZF type=del 2024-05-09T20:17:07.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.475+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=KHFGLLCTA2KH type=del size=4004578 2024-05-09T20:17:07.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.479+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=ZCFDEVNFS8VI type=del 2024-05-09T20:17:07.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.483+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~F33NE7IKWI3O.0 size=1048576 2024-05-09T20:17:07.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.487+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~F33NE7IKWI3O.1 size=156757 2024-05-09T20:17:07.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.487+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=F33NE7IKWI3O tag=5KU3TOCQZQS7 type=write size=2253909 2024-05-09T20:17:07.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.491+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=15V3H2E6W9N5 type=del 2024-05-09T20:17:07.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.495+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=9P25F9Y27PQ1 type=del size=2009498 2024-05-09T20:17:07.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.495+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=O04X15MTTEH5 type=del 2024-05-09T20:17:07.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.499+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=PHEAIRI3QHZC type=del size=216195 2024-05-09T20:17:07.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.503+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~D5HLYOIL1ETL.0 size=911598 2024-05-09T20:17:07.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.515+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=D5HLYOIL1ETL tag=U0XRVIKZ2E4B type=write size=1960174 2024-05-09T20:17:07.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.519+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=JO2WJ2LO0IPQ type=del size=2155551 2024-05-09T20:17:07.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.519+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=KI6W1C9KNWJ6 type=del 2024-05-09T20:17:07.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.523+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=9XTF2STD6M1K type=write size=201030 2024-05-09T20:17:07.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.527+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~5JQ70JYXJW3I.0 size=1048576 2024-05-09T20:17:07.523 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.531+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~5JQ70JYXJW3I.1 size=1048576 2024-05-09T20:17:07.523 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.531+0000 7fdc882e58c0 1 < canceled part key=_multipart_SPXHCWZICCWY.2~5JQ70JYXJW3I.2 size=577090 2024-05-09T20:17:07.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.535+0000 7fdc882e58c0 1 < canceled multipart upload key=SPXHCWZICCWY upload=5JQ70JYXJW3I tag=HAF758W6QRSJ type=write size=3722818 2024-05-09T20:17:07.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.539+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=4MFWXAEI6IBO type=del size=3612125 2024-05-09T20:17:07.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.543+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=P1U1W99G2LK8 type=del 2024-05-09T20:17:07.543 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.551+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~8ITYIKBP8NPJ.0 size=1048576 2024-05-09T20:17:07.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.551+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~8ITYIKBP8NPJ.1 size=714379 2024-05-09T20:17:07.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.551+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=8ITYIKBP8NPJ tag=2UI3FCIBVM78 type=write size=2811531 2024-05-09T20:17:07.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.555+0000 7fdc882e58c0 1 < canceled operation key=EIMPFFK4I6NZ tag=IJQ2OHEO9SXG type=del size=3404024 2024-05-09T20:17:07.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.559+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=JB50XCXD9UV2 type=del size=1846336 2024-05-09T20:17:07.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.563+0000 7fdc882e58c0 1 < canceled operation key=NYHR6GTPQYZS tag=I1CAR18YL1EX type=del size=2444958 2024-05-09T20:17:07.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.567+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~G2L21XPO95IE.0 size=1048576 2024-05-09T20:17:07.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.571+0000 7fdc882e58c0 1 > prepared part key=_multipart_NBAECV9L85UK.2~G2L21XPO95IE.1 size=493766 2024-05-09T20:17:07.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.571+0000 7fdc882e58c0 1 > prepared multipart upload key=NBAECV9L85UK upload=G2L21XPO95IE tag=C84V6L6YMME5 type=write size=2590918 2024-05-09T20:17:07.568 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.575+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=D93V1D6W6LTY type=del 2024-05-09T20:17:07.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.579+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=PJUHQQUTQN3R type=del size=3616587 2024-05-09T20:17:07.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.583+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=56FIBUSQ0ZQ4 type=del 2024-05-09T20:17:07.582 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.587+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~O07WVX8624A2.0 size=1048576 2024-05-09T20:17:07.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.591+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~O07WVX8624A2.1 size=811914 2024-05-09T20:17:07.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.591+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=O07WVX8624A2 tag=6VX8X32B17M9 type=write size=2909066 2024-05-09T20:17:07.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.595+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~92QGYOSLK7UX.0 size=138332 2024-05-09T20:17:07.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.599+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=92QGYOSLK7UX tag=O08SSXQC5WRY type=write size=1186908 2024-05-09T20:17:07.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.603+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=HARVSQ13FEIB type=del size=1991953 2024-05-09T20:17:07.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.607+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=9ZJ63RHYDUAA type=del 2024-05-09T20:17:07.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.611+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=9RR8WBMEU09W type=del size=1521784 2024-05-09T20:17:07.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.611+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~FWO7YZSOA38E.0 size=1048576 2024-05-09T20:17:07.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.615+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~FWO7YZSOA38E.1 size=246827 2024-05-09T20:17:07.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.619+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=FWO7YZSOA38E tag=LGB1C04VJBGP type=write size=2343979 2024-05-09T20:17:07.616 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.623+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=186FMAY4BGWB type=del size=1852550 2024-05-09T20:17:07.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.627+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~LYLVLYCPYV0W.0 size=1048576 2024-05-09T20:17:07.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.631+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~LYLVLYCPYV0W.1 size=1048576 2024-05-09T20:17:07.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.631+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~LYLVLYCPYV0W.2 size=924888 2024-05-09T20:17:07.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.639+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=LYLVLYCPYV0W tag=69SFOR6GD2JV type=write size=4070616 2024-05-09T20:17:07.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.643+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=QFWXRWFD5EUB type=del size=1484879 2024-05-09T20:17:07.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.647+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=KHFGLLCTA2KH type=del 2024-05-09T20:17:07.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.651+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~BL42X3ED5IWX.0 size=1048576 2024-05-09T20:17:07.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.655+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~BL42X3ED5IWX.1 size=753287 2024-05-09T20:17:07.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.655+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=BL42X3ED5IWX tag=A16S047HVNEB type=write size=2850439 2024-05-09T20:17:07.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.659+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~F33NE7IKWI3O.0 size=1048576 2024-05-09T20:17:07.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.663+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~F33NE7IKWI3O.1 size=156757 2024-05-09T20:17:07.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.667+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=F33NE7IKWI3O tag=5KU3TOCQZQS7 type=write size=2253909 2024-05-09T20:17:07.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.671+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=YDC79EIULW7D type=write size=865522 2024-05-09T20:17:07.667 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.675+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=9P25F9Y27PQ1 type=del 2024-05-09T20:17:07.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.679+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=W8IT6S7SMD45 type=del size=2365162 2024-05-09T20:17:07.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.683+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=PHEAIRI3QHZC type=del 2024-05-09T20:17:07.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.687+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~GLYIXRAZRH36.0 size=1048576 2024-05-09T20:17:07.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.691+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~GLYIXRAZRH36.1 size=1048576 2024-05-09T20:17:07.685 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.691+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~GLYIXRAZRH36.2 size=668699 2024-05-09T20:17:07.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.691+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=GLYIXRAZRH36 tag=2I1P2I79AZ2H type=write size=3814427 2024-05-09T20:17:07.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.695+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=JO2WJ2LO0IPQ type=del 2024-05-09T20:17:07.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.699+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=3KAT0O9ZJUGL type=del size=2438642 2024-05-09T20:17:07.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.703+0000 7fdc882e58c0 1 < completed write operation key=TQ5ZHATLNVPP tag=9XTF2STD6M1K type=write size=201030 2024-05-09T20:17:07.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.707+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=7OTJC9OZZPF9 type=del size=704173 2024-05-09T20:17:07.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.711+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=4MFWXAEI6IBO type=del size=3612125 2024-05-09T20:17:07.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.715+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=CP57ZDNEWHPG type=del size=284924 2024-05-09T20:17:07.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.719+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~8ITYIKBP8NPJ.0 size=1048576 2024-05-09T20:17:07.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.723+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~8ITYIKBP8NPJ.1 size=714379 2024-05-09T20:17:07.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.727+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=8ITYIKBP8NPJ tag=2UI3FCIBVM78 type=write size=2811531 2024-05-09T20:17:07.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.731+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=8FQYMQLCGS2G type=del size=838237 2024-05-09T20:17:07.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.735+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=JB50XCXD9UV2 type=del 2024-05-09T20:17:07.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.739+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=0XZFELXLGEOA type=del size=282017 2024-05-09T20:17:07.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.743+0000 7fdc882e58c0 1 < completed part key=_multipart_NBAECV9L85UK.2~G2L21XPO95IE.0 size=1048576 2024-05-09T20:17:07.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.743+0000 7fdc882e58c0 1 < canceled part key=_multipart_NBAECV9L85UK.2~G2L21XPO95IE.1 size=493766 2024-05-09T20:17:07.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.747+0000 7fdc882e58c0 1 < canceled multipart upload key=NBAECV9L85UK upload=G2L21XPO95IE tag=C84V6L6YMME5 type=write size=2590918 2024-05-09T20:17:07.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.751+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=MVNE1423UR4E type=del size=3763172 2024-05-09T20:17:07.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.751+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=PJUHQQUTQN3R type=del 2024-05-09T20:17:07.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.755+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=UDBUQMKFP0C4 type=del size=2731671 2024-05-09T20:17:07.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.759+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~O07WVX8624A2.0 size=1048576 2024-05-09T20:17:07.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.763+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~O07WVX8624A2.1 size=811914 2024-05-09T20:17:07.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.767+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=O07WVX8624A2 tag=6VX8X32B17M9 type=write size=2909066 2024-05-09T20:17:07.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.767+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=M6KUN8RAI057 type=del size=1537312 2024-05-09T20:17:07.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.775+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=HARVSQ13FEIB type=del size=1991953 2024-05-09T20:17:07.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.779+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~Q076TUK67QTQ.0 size=1048576 2024-05-09T20:17:07.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.783+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~Q076TUK67QTQ.1 size=528911 2024-05-09T20:17:07.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.783+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=Q076TUK67QTQ tag=QNVMETABX512 type=write size=2626063 2024-05-09T20:17:07.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.787+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=9RR8WBMEU09W type=del 2024-05-09T20:17:07.782 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.791+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=99IFR12PE9VY type=write size=447155 2024-05-09T20:17:07.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.795+0000 7fdc882e58c0 1 < suggested remove operation key=54R9XPO62J0M tag=186FMAY4BGWB type=del 2024-05-09T20:17:07.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.799+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=DT9K3MHV6MVJ type=write size=702219 2024-05-09T20:17:07.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.803+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=QFWXRWFD5EUB type=del 2024-05-09T20:17:07.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.807+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=5R8S6S1GHZ02 type=del size=3071762 2024-05-09T20:17:07.802 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.811+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~BL42X3ED5IWX.0 size=1048576 2024-05-09T20:17:07.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.811+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~BL42X3ED5IWX.1 size=753287 2024-05-09T20:17:07.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.815+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=BL42X3ED5IWX tag=A16S047HVNEB type=write size=2850439 2024-05-09T20:17:07.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.819+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=N3UI0OJMNW6B type=del size=822066 2024-05-09T20:17:07.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.823+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=YDC79EIULW7D type=write size=865522 2024-05-09T20:17:07.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.827+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=GH95SCDHW1BN type=del size=525017 2024-05-09T20:17:07.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.831+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=W8IT6S7SMD45 type=del 2024-05-09T20:17:07.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.835+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=JRF4LO0GF5B2 type=del size=3849434 2024-05-09T20:17:07.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.835+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~GLYIXRAZRH36.0 size=1048576 2024-05-09T20:17:07.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.839+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~GLYIXRAZRH36.1 size=1048576 2024-05-09T20:17:07.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.839+0000 7fdc882e58c0 1 < canceled part key=_multipart_CNV7NHP27JZ7.2~GLYIXRAZRH36.2 size=668699 2024-05-09T20:17:07.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.843+0000 7fdc882e58c0 1 < canceled multipart upload key=CNV7NHP27JZ7 upload=GLYIXRAZRH36 tag=2I1P2I79AZ2H type=write size=3814427 2024-05-09T20:17:07.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.847+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~I371CVH87BHV.0 size=1048576 2024-05-09T20:17:07.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.851+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~I371CVH87BHV.1 size=1048576 2024-05-09T20:17:07.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.851+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~I371CVH87BHV.2 size=804565 2024-05-09T20:17:07.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.851+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=I371CVH87BHV tag=HJ96RN3IMN5M type=write size=3950293 2024-05-09T20:17:07.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.855+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=3KAT0O9ZJUGL type=del 2024-05-09T20:17:07.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.859+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~XUFI0QDN4JZT.0 size=1048576 2024-05-09T20:17:07.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.863+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~XUFI0QDN4JZT.1 size=454872 2024-05-09T20:17:07.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.863+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=XUFI0QDN4JZT tag=BD0ZIH8I8KHX type=write size=2552024 2024-05-09T20:17:07.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.867+0000 7fdc882e58c0 1 < suggested remove operation key=346HV1XQKCOZ tag=7OTJC9OZZPF9 type=del 2024-05-09T20:17:07.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.871+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=60J48S5LU9WV type=del size=1236334 2024-05-09T20:17:07.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.871+0000 7fdc882e58c0 -1 < failed to complete operation key=TQ5ZHATLNVPP tag=CP57ZDNEWHPG type=del size=284924: (22) Invalid argument 2024-05-09T20:17:07.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.875+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=M7G7KWS57IA9 type=del size=3597060 2024-05-09T20:17:07.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.879+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=8FQYMQLCGS2G type=del 2024-05-09T20:17:07.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.883+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=CBIWZRSEQUW7 type=del size=2269216 2024-05-09T20:17:07.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.891+0000 7fdc882e58c0 1 < suggested remove operation key=IZPPFLWVPRKO tag=0XZFELXLGEOA type=del 2024-05-09T20:17:07.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.891+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=58SZE4N6QVEP type=del size=2679253 2024-05-09T20:17:07.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.895+0000 7fdc882e58c0 1 < canceled operation key=OGVA2778MJWU tag=MVNE1423UR4E type=del size=3763172 2024-05-09T20:17:07.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.903+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~4UZ0CCL4I43Z.0 size=1048576 2024-05-09T20:17:07.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.907+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~4UZ0CCL4I43Z.1 size=334006 2024-05-09T20:17:07.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.907+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=4UZ0CCL4I43Z tag=XSHCPRZBPZ8W type=write size=2431158 2024-05-09T20:17:07.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.907+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=UDBUQMKFP0C4 type=del 2024-05-09T20:17:07.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.911+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=ZYJNMNCEACDL type=write size=291172 2024-05-09T20:17:07.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.915+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=M6KUN8RAI057 type=del 2024-05-09T20:17:07.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.919+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~V6N1DVB0GW6X.0 size=1048576 2024-05-09T20:17:07.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.923+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~V6N1DVB0GW6X.1 size=1048576 2024-05-09T20:17:07.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.923+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~V6N1DVB0GW6X.2 size=284468 2024-05-09T20:17:07.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.923+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=V6N1DVB0GW6X tag=23OESJB340C6 type=write size=3430196 2024-05-09T20:17:07.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.927+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~Q076TUK67QTQ.0 size=1048576 2024-05-09T20:17:07.922 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.931+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~Q076TUK67QTQ.1 size=528911 2024-05-09T20:17:07.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.935+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=Q076TUK67QTQ tag=QNVMETABX512 type=write size=2626063 2024-05-09T20:17:07.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.939+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=0V7U3S2W58YD type=del size=436735 2024-05-09T20:17:07.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.939+0000 7fdc882e58c0 1 < completed write operation key=SPXHCWZICCWY tag=99IFR12PE9VY type=write size=447155 2024-05-09T20:17:07.936 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.943+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=MM336OSQ141X type=del size=594136 2024-05-09T20:17:07.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.951+0000 7fdc882e58c0 1 < suggested update operation key=G0DX1F257RLD tag=DT9K3MHV6MVJ type=write size=702219 2024-05-09T20:17:07.945 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.951+0000 7fdc882e58c0 1 > prepared operation key=346HV1XQKCOZ tag=YGM42WHU1LIJ type=del size=3866828 2024-05-09T20:17:07.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.955+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=5R8S6S1GHZ02 type=del 2024-05-09T20:17:07.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.959+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=NLSDZWRE4JHP type=del size=1185567 2024-05-09T20:17:07.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.963+0000 7fdc882e58c0 1 < completed delete operation key=J2Q86HKRMAKE tag=N3UI0OJMNW6B type=del 2024-05-09T20:17:07.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.967+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~D70ZYRYUOZE9.0 size=1048576 2024-05-09T20:17:07.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.967+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~D70ZYRYUOZE9.1 size=1048576 2024-05-09T20:17:07.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.971+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~D70ZYRYUOZE9.2 size=763270 2024-05-09T20:17:07.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.971+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=D70ZYRYUOZE9 tag=KS3CBJ8V1TNY type=write size=3908998 2024-05-09T20:17:07.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.975+0000 7fdc882e58c0 1 < suggested update operation key=P2DNA4DGI54P tag=GH95SCDHW1BN type=del size=525017 2024-05-09T20:17:07.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.983+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~WEEXIILU6H3O.0 size=240173 2024-05-09T20:17:07.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.983+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=WEEXIILU6H3O tag=GNG1U1UQIOU9 type=write size=1288749 2024-05-09T20:17:07.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.983+0000 7fdc882e58c0 -1 < failed to complete operation key=P2DNA4DGI54P tag=JRF4LO0GF5B2 type=del size=3849434: (22) Invalid argument 2024-05-09T20:17:07.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.987+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=349SHGFPRZBE type=del size=3618573 2024-05-09T20:17:07.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.991+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~I371CVH87BHV.0 size=1048576 2024-05-09T20:17:07.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.991+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~I371CVH87BHV.1 size=1048576 2024-05-09T20:17:07.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.995+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~I371CVH87BHV.2 size=804565 2024-05-09T20:17:07.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:07.999+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=I371CVH87BHV tag=HJ96RN3IMN5M type=write size=3950293 2024-05-09T20:17:07.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.003+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=1N2KZ1KVUAYU type=del size=3348894 2024-05-09T20:17:07.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.003+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~XUFI0QDN4JZT.0 size=1048576 2024-05-09T20:17:07.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.007+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~XUFI0QDN4JZT.1 size=454872 2024-05-09T20:17:08.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.011+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=XUFI0QDN4JZT tag=BD0ZIH8I8KHX type=write size=2552024 2024-05-09T20:17:08.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.015+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=R497A4WM0ZA5 type=del size=4170842 2024-05-09T20:17:08.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.015+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=60J48S5LU9WV type=del 2024-05-09T20:17:08.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.023+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~PVFQA6WLOM8Y.0 size=1048576 2024-05-09T20:17:08.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.023+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~PVFQA6WLOM8Y.1 size=1048576 2024-05-09T20:17:08.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.027+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~PVFQA6WLOM8Y.2 size=118881 2024-05-09T20:17:08.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.027+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=PVFQA6WLOM8Y tag=I4DIMDLZZ8KG type=write size=3264609 2024-05-09T20:17:08.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.031+0000 7fdc882e58c0 1 < canceled operation key=8FFZZDURUWM1 tag=M7G7KWS57IA9 type=del size=3597060 2024-05-09T20:17:08.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.031+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=8HLDF797J9TV type=del size=631807 2024-05-09T20:17:08.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.035+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=CBIWZRSEQUW7 type=del 2024-05-09T20:17:08.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.039+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=M45W4GRZQB8F type=write size=906983 2024-05-09T20:17:08.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.043+0000 7fdc882e58c0 1 < suggested remove operation key=SPXHCWZICCWY tag=58SZE4N6QVEP type=del 2024-05-09T20:17:08.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.047+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=9D24CB2NABBU type=del size=2481148 2024-05-09T20:17:08.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.051+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~4UZ0CCL4I43Z.0 size=1048576 2024-05-09T20:17:08.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.055+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~4UZ0CCL4I43Z.1 size=334006 2024-05-09T20:17:08.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.059+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=4UZ0CCL4I43Z tag=XSHCPRZBPZ8W type=write size=2431158 2024-05-09T20:17:08.054 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.063+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=F5UCDWW228Z5 type=del size=2288287 2024-05-09T20:17:08.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.063+0000 7fdc882e58c0 -1 < failed to complete operation key=G0DX1F257RLD tag=ZYJNMNCEACDL type=write size=291172: (22) Invalid argument 2024-05-09T20:17:08.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.067+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=LUBX7HJ5Z7FW type=write size=194214 2024-05-09T20:17:08.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.071+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~V6N1DVB0GW6X.0 size=1048576 2024-05-09T20:17:08.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.075+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~V6N1DVB0GW6X.1 size=1048576 2024-05-09T20:17:08.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.075+0000 7fdc882e58c0 1 < canceled part key=_multipart_IZPPFLWVPRKO.2~V6N1DVB0GW6X.2 size=284468 2024-05-09T20:17:08.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.079+0000 7fdc882e58c0 1 < canceled multipart upload key=IZPPFLWVPRKO upload=V6N1DVB0GW6X tag=23OESJB340C6 type=write size=3430196 2024-05-09T20:17:08.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.083+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=K4N2Q13PPDMC type=del size=2200334 2024-05-09T20:17:08.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.087+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=0V7U3S2W58YD type=del 2024-05-09T20:17:08.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.087+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=IN692AL1V728 type=write size=746825 2024-05-09T20:17:08.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.091+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=MM336OSQ141X type=del 2024-05-09T20:17:08.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.095+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=LERT3DINJKZ9 type=del size=1840008 2024-05-09T20:17:08.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.099+0000 7fdc882e58c0 1 < completed delete operation key=346HV1XQKCOZ tag=YGM42WHU1LIJ type=del 2024-05-09T20:17:08.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.099+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=N4WG8UM3QBJP type=write size=855026 2024-05-09T20:17:08.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.107+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=NLSDZWRE4JHP type=del 2024-05-09T20:17:08.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.111+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=YILB5XFJ6XOQ type=del size=1968944 2024-05-09T20:17:08.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.115+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~D70ZYRYUOZE9.0 size=1048576 2024-05-09T20:17:08.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.119+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~D70ZYRYUOZE9.1 size=1048576 2024-05-09T20:17:08.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.119+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~D70ZYRYUOZE9.2 size=763270 2024-05-09T20:17:08.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.123+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=D70ZYRYUOZE9 tag=KS3CBJ8V1TNY type=write size=3908998 2024-05-09T20:17:08.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.127+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=CNTKYDAOH475 type=del size=3148610 2024-05-09T20:17:08.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.131+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~WEEXIILU6H3O.0 size=240173 2024-05-09T20:17:08.127 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.135+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=WEEXIILU6H3O tag=GNG1U1UQIOU9 type=write size=1288749 2024-05-09T20:17:08.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.139+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=B2MBJ1FHIWID type=del size=3760273 2024-05-09T20:17:08.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.139+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=349SHGFPRZBE type=del 2024-05-09T20:17:08.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.143+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=PJNBV2NSFQKU type=write size=1008617 2024-05-09T20:17:08.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.147+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=1N2KZ1KVUAYU type=del 2024-05-09T20:17:08.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.147+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=AWOGMIZQ5L8T type=del size=3368814 2024-05-09T20:17:08.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.151+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=R497A4WM0ZA5 type=del 2024-05-09T20:17:08.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.155+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~9LXKL7WM8UIP.0 size=1048576 2024-05-09T20:17:08.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.159+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~9LXKL7WM8UIP.1 size=217626 2024-05-09T20:17:08.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.159+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=9LXKL7WM8UIP tag=8MJVZAXDJCWM type=write size=2314778 2024-05-09T20:17:08.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.159+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~PVFQA6WLOM8Y.0 size=1048576 2024-05-09T20:17:08.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.163+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~PVFQA6WLOM8Y.1 size=1048576 2024-05-09T20:17:08.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.167+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~PVFQA6WLOM8Y.2 size=118881 2024-05-09T20:17:08.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.171+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=PVFQA6WLOM8Y tag=I4DIMDLZZ8KG type=write size=3264609 2024-05-09T20:17:08.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.171+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=AHT0H6WUW40P type=del size=2974355 2024-05-09T20:17:08.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.179+0000 7fdc882e58c0 1 < suggested remove operation key=J2Q86HKRMAKE tag=8HLDF797J9TV type=del 2024-05-09T20:17:08.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.183+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~1OP0P72N4BJT.0 size=1048576 2024-05-09T20:17:08.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.187+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~1OP0P72N4BJT.1 size=1048576 2024-05-09T20:17:08.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.187+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~1OP0P72N4BJT.2 size=538636 2024-05-09T20:17:08.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.187+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=1OP0P72N4BJT tag=1J40OE50OJTV type=write size=3684364 2024-05-09T20:17:08.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.191+0000 7fdc882e58c0 1 < completed write operation key=NYHR6GTPQYZS tag=M45W4GRZQB8F type=write size=906983 2024-05-09T20:17:08.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.191+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=O53Z40O2X3Z2 type=del size=3379802 2024-05-09T20:17:08.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.195+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=9D24CB2NABBU type=del 2024-05-09T20:17:08.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.199+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=R0X6XOX682SQ type=del size=3708126 2024-05-09T20:17:08.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.203+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=F5UCDWW228Z5 type=del 2024-05-09T20:17:08.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.207+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~VBY2GZXQCK0C.0 size=1041617 2024-05-09T20:17:08.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.207+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=VBY2GZXQCK0C tag=V0D5HB0TY5DL type=write size=2090193 2024-05-09T20:17:08.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.207+0000 7fdc882e58c0 -1 < no bi entry to suggest for operation key=J2Q86HKRMAKE tag=LUBX7HJ5Z7FW type=write size=194214: (2) No such file or directory 2024-05-09T20:17:08.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.211+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~8H2G75Y1IZ6Z.0 size=1048576 2024-05-09T20:17:08.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.215+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~8H2G75Y1IZ6Z.1 size=754466 2024-05-09T20:17:08.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.215+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=8H2G75Y1IZ6Z tag=LD5ZT52BV58F type=write size=2851618 2024-05-09T20:17:08.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.215+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=K4N2Q13PPDMC type=del 2024-05-09T20:17:08.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.223+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~63MOOQMNTB6F.0 size=1048576 2024-05-09T20:17:08.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.223+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~63MOOQMNTB6F.1 size=1048576 2024-05-09T20:17:08.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.227+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~63MOOQMNTB6F.2 size=134803 2024-05-09T20:17:08.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.227+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=63MOOQMNTB6F tag=UQIFDJ8I7ZHU type=write size=3280531 2024-05-09T20:17:08.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.231+0000 7fdc882e58c0 1 < completed write operation key=TQ5ZHATLNVPP tag=IN692AL1V728 type=write size=746825 2024-05-09T20:17:08.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.235+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~90XLT1VRO1PV.0 size=1048576 2024-05-09T20:17:08.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.235+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~90XLT1VRO1PV.1 size=82388 2024-05-09T20:17:08.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.235+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=90XLT1VRO1PV tag=53PNTPM5M1T9 type=write size=2179540 2024-05-09T20:17:08.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.239+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=LERT3DINJKZ9 type=del 2024-05-09T20:17:08.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.243+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~986O2DAQFA09.0 size=299325 2024-05-09T20:17:08.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.243+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=986O2DAQFA09 tag=MVYXR20TQGNX type=write size=1347901 2024-05-09T20:17:08.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.247+0000 7fdc882e58c0 1 < completed write operation key=OGVA2778MJWU tag=N4WG8UM3QBJP type=write size=855026 2024-05-09T20:17:08.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.251+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=QIFPZZRPH3S6 type=del size=3852749 2024-05-09T20:17:08.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.255+0000 7fdc882e58c0 1 < suggested update operation key=QM53IENH9CX4 tag=YILB5XFJ6XOQ type=del size=1968944 2024-05-09T20:17:08.254 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.259+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~8DU9EX2ALVVT.0 size=1048576 2024-05-09T20:17:08.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.263+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~8DU9EX2ALVVT.1 size=711912 2024-05-09T20:17:08.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.263+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=8DU9EX2ALVVT tag=O7T1KFO9S7YV type=write size=2809064 2024-05-09T20:17:08.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.271+0000 7fdc882e58c0 1 < suggested remove operation key=G0DX1F257RLD tag=CNTKYDAOH475 type=del 2024-05-09T20:17:08.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.275+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~AD5W2QOOJ6IX.0 size=1048576 2024-05-09T20:17:08.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.275+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~AD5W2QOOJ6IX.1 size=712667 2024-05-09T20:17:08.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.275+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=AD5W2QOOJ6IX tag=K8YCN02HRJ4G type=write size=2809819 2024-05-09T20:17:08.275 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.283+0000 7fdc882e58c0 1 < suggested remove operation key=TWWOL5D8C1MQ tag=B2MBJ1FHIWID type=del 2024-05-09T20:17:08.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.287+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=E0ZB33BIBSF5 type=del size=4055696 2024-05-09T20:17:08.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.287+0000 7fdc882e58c0 1 < completed write operation key=5H2GWM9LQELZ tag=PJNBV2NSFQKU type=write size=1008617 2024-05-09T20:17:08.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.291+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=L72GZ3NC9RZ5 type=del size=2569236 2024-05-09T20:17:08.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.295+0000 7fdc882e58c0 1 < canceled operation key=TQ5ZHATLNVPP tag=AWOGMIZQ5L8T type=del size=3368814 2024-05-09T20:17:08.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.299+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=DUJ7L35U32YZ type=del size=106126 2024-05-09T20:17:08.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.303+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~9LXKL7WM8UIP.0 size=1048576 2024-05-09T20:17:08.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.303+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~9LXKL7WM8UIP.1 size=217626 2024-05-09T20:17:08.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.311+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=9LXKL7WM8UIP tag=8MJVZAXDJCWM type=write size=2314778 2024-05-09T20:17:08.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.315+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~72G2GRYFG1M2.0 size=1048576 2024-05-09T20:17:08.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.319+0000 7fdc882e58c0 1 > prepared part key=_multipart_8FFZZDURUWM1.2~72G2GRYFG1M2.1 size=716755 2024-05-09T20:17:08.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.319+0000 7fdc882e58c0 1 > prepared multipart upload key=8FFZZDURUWM1 upload=72G2GRYFG1M2 tag=2ENX1WNNPV0E type=write size=2813907 2024-05-09T20:17:08.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.323+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=AHT0H6WUW40P type=del 2024-05-09T20:17:08.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.327+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=AV9ISZSIIIA7 type=del size=1805074 2024-05-09T20:17:08.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.327+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~1OP0P72N4BJT.0 size=1048576 2024-05-09T20:17:08.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.327+0000 7fdc882e58c0 1 < canceled part key=_multipart_BS0T1OQOBS67.2~1OP0P72N4BJT.1 size=1048576 2024-05-09T20:17:08.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.331+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~1OP0P72N4BJT.2 size=538636 2024-05-09T20:17:08.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.339+0000 7fdc882e58c0 1 < canceled multipart upload key=BS0T1OQOBS67 upload=1OP0P72N4BJT tag=1J40OE50OJTV type=write size=3684364 2024-05-09T20:17:08.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.343+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~QGJGGG5R3RQ6.0 size=1048576 2024-05-09T20:17:08.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.347+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~QGJGGG5R3RQ6.1 size=1048576 2024-05-09T20:17:08.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.347+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~QGJGGG5R3RQ6.2 size=686664 2024-05-09T20:17:08.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.347+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=QGJGGG5R3RQ6 tag=I6TWMV4XA24I type=write size=3832392 2024-05-09T20:17:08.343 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.351+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=O53Z40O2X3Z2 type=del 2024-05-09T20:17:08.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.355+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=347DW57KBXSF type=del size=389407 2024-05-09T20:17:08.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.359+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=R0X6XOX682SQ type=del 2024-05-09T20:17:08.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.359+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=UHCGDGXII557 type=del size=1027495 2024-05-09T20:17:08.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.363+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~VBY2GZXQCK0C.0 size=1041617 2024-05-09T20:17:08.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.371+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=VBY2GZXQCK0C tag=V0D5HB0TY5DL type=write size=2090193 2024-05-09T20:17:08.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.371+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=SU2C5B2I9J17 type=del size=2138384 2024-05-09T20:17:08.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.375+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~8H2G75Y1IZ6Z.0 size=1048576 2024-05-09T20:17:08.371 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.379+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~8H2G75Y1IZ6Z.1 size=754466 2024-05-09T20:17:08.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.387+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=8H2G75Y1IZ6Z tag=LD5ZT52BV58F type=write size=2851618 2024-05-09T20:17:08.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.395+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~UZ84BMBIWFXU.0 size=92365 2024-05-09T20:17:08.450 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.395+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=UZ84BMBIWFXU tag=S50A6C5IQW99 type=write size=1140941 2024-05-09T20:17:08.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.399+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~63MOOQMNTB6F.0 size=1048576 2024-05-09T20:17:08.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.403+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~63MOOQMNTB6F.1 size=1048576 2024-05-09T20:17:08.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.403+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~63MOOQMNTB6F.2 size=134803 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.411+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=63MOOQMNTB6F tag=UQIFDJ8I7ZHU type=write size=3280531 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.415+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=83CLYK1BSXXW type=del size=320352 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.419+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~90XLT1VRO1PV.0 size=1048576 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.423+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~90XLT1VRO1PV.1 size=82388 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.427+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=90XLT1VRO1PV tag=53PNTPM5M1T9 type=write size=2179540 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.431+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=MFS489SOW0A0 type=del size=192853 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.431+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~986O2DAQFA09.0 size=299325 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.435+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=986O2DAQFA09 tag=MVYXR20TQGNX type=write size=1347901 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.443+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~ZXUF3JP6UAW6.0 size=1048576 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.443+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~ZXUF3JP6UAW6.1 size=1048576 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.447+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~ZXUF3JP6UAW6.2 size=286762 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.447+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=ZXUF3JP6UAW6 tag=KTPD5XOXKEE0 type=write size=3432490 2024-05-09T20:17:08.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.447+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=QIFPZZRPH3S6 type=del 2024-05-09T20:17:08.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.455+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~QQUAP87IHRP9.0 size=1048576 2024-05-09T20:17:08.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.455+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~QQUAP87IHRP9.1 size=2358 2024-05-09T20:17:08.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.455+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=QQUAP87IHRP9 tag=ZTD8RXZ6JVPD type=write size=2099510 2024-05-09T20:17:08.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.459+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~8DU9EX2ALVVT.0 size=1048576 2024-05-09T20:17:08.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.459+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~8DU9EX2ALVVT.1 size=711912 2024-05-09T20:17:08.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.467+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=8DU9EX2ALVVT tag=O7T1KFO9S7YV type=write size=2809064 2024-05-09T20:17:08.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.467+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=9U66P3P2ZS15 type=del size=4094197 2024-05-09T20:17:08.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.471+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~AD5W2QOOJ6IX.0 size=1048576 2024-05-09T20:17:08.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.475+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~AD5W2QOOJ6IX.1 size=712667 2024-05-09T20:17:08.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.479+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=AD5W2QOOJ6IX tag=K8YCN02HRJ4G type=write size=2809819 2024-05-09T20:17:08.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.483+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~B17Z44Z4SUC2.0 size=1048576 2024-05-09T20:17:08.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.487+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~B17Z44Z4SUC2.1 size=521124 2024-05-09T20:17:08.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.487+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=B17Z44Z4SUC2 tag=X8MSSE1GGJPE type=write size=2618276 2024-05-09T20:17:08.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.491+0000 7fdc882e58c0 1 < suggested remove operation key=NBAECV9L85UK tag=E0ZB33BIBSF5 type=del 2024-05-09T20:17:08.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.499+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~F7WN3CJ3Y1JB.0 size=523451 2024-05-09T20:17:08.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.499+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=F7WN3CJ3Y1JB tag=2MXQHPX8CJJ9 type=write size=1572027 2024-05-09T20:17:08.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.499+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=L72GZ3NC9RZ5 type=del 2024-05-09T20:17:08.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.507+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~ALUFL3SFIJOR.0 size=1048576 2024-05-09T20:17:08.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.507+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~ALUFL3SFIJOR.1 size=1048576 2024-05-09T20:17:08.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.511+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~ALUFL3SFIJOR.2 size=489267 2024-05-09T20:17:08.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.511+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=ALUFL3SFIJOR tag=79MVYHXLXQO5 type=write size=3634995 2024-05-09T20:17:08.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.515+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=DUJ7L35U32YZ type=del 2024-05-09T20:17:08.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.519+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~90REFP1FZRLD.0 size=1048576 2024-05-09T20:17:08.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.523+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~90REFP1FZRLD.1 size=49626 2024-05-09T20:17:08.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.523+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=90REFP1FZRLD tag=XOAR1RZL65Q3 type=write size=2146778 2024-05-09T20:17:08.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.523+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~72G2GRYFG1M2.0 size=1048576 2024-05-09T20:17:08.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.527+0000 7fdc882e58c0 1 < completed part key=_multipart_8FFZZDURUWM1.2~72G2GRYFG1M2.1 size=716755 2024-05-09T20:17:08.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.531+0000 7fdc882e58c0 1 < completed multipart upload key=8FFZZDURUWM1 upload=72G2GRYFG1M2 tag=2ENX1WNNPV0E type=write size=2813907 2024-05-09T20:17:08.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.535+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~X51FNOXTH9Y9.0 size=1048576 2024-05-09T20:17:08.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.539+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~X51FNOXTH9Y9.1 size=1048576 2024-05-09T20:17:08.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.543+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~X51FNOXTH9Y9.2 size=170124 2024-05-09T20:17:08.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.543+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=X51FNOXTH9Y9 tag=1D20W9MB0QUO type=write size=3315852 2024-05-09T20:17:08.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.543+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=AV9ISZSIIIA7 type=del 2024-05-09T20:17:08.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.547+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=2KO6D9APSCCG type=del size=2957004 2024-05-09T20:17:08.543 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.551+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~QGJGGG5R3RQ6.0 size=1048576 2024-05-09T20:17:08.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.555+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~QGJGGG5R3RQ6.1 size=1048576 2024-05-09T20:17:08.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.555+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~QGJGGG5R3RQ6.2 size=686664 2024-05-09T20:17:08.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.563+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=QGJGGG5R3RQ6 tag=I6TWMV4XA24I type=write size=3832392 2024-05-09T20:17:08.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.567+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~N38DAW32G7WB.0 size=51815 2024-05-09T20:17:08.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.567+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=N38DAW32G7WB tag=9148MQCKZSY0 type=write size=1100391 2024-05-09T20:17:08.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.571+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=347DW57KBXSF type=del 2024-05-09T20:17:08.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.575+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=YX1SQZUBW59C type=del size=718975 2024-05-09T20:17:08.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.579+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=UHCGDGXII557 type=del 2024-05-09T20:17:08.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.583+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=LKB7154E9LR9 type=del size=3556474 2024-05-09T20:17:08.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.587+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=SU2C5B2I9J17 type=del 2024-05-09T20:17:08.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.587+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=8DQ17LBEBZ1J type=del size=917442 2024-05-09T20:17:08.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.591+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~UZ84BMBIWFXU.0 size=92365 2024-05-09T20:17:08.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.595+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=UZ84BMBIWFXU tag=S50A6C5IQW99 type=write size=1140941 2024-05-09T20:17:08.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.599+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=A1MCRU4H413R type=del size=836529 2024-05-09T20:17:08.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.603+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=83CLYK1BSXXW type=del 2024-05-09T20:17:08.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.607+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~9FRTGW5XUQZJ.0 size=1048576 2024-05-09T20:17:08.603 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.611+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~9FRTGW5XUQZJ.1 size=1048576 2024-05-09T20:17:08.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.611+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~9FRTGW5XUQZJ.2 size=1024858 2024-05-09T20:17:08.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.611+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=9FRTGW5XUQZJ tag=B71N19HJ7INJ type=write size=4170586 2024-05-09T20:17:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.615+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=MFS489SOW0A0 type=del 2024-05-09T20:17:08.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.619+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=4AN3KFPRNC86 type=del size=2785272 2024-05-09T20:17:08.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.619+0000 7fdc882e58c0 1 < canceled part key=_multipart_O07UHAIJZF0R.2~ZXUF3JP6UAW6.0 size=1048576 2024-05-09T20:17:08.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.623+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~ZXUF3JP6UAW6.1 size=1048576 2024-05-09T20:17:08.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.623+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~ZXUF3JP6UAW6.2 size=286762 2024-05-09T20:17:08.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.631+0000 7fdc882e58c0 1 < canceled multipart upload key=O07UHAIJZF0R upload=ZXUF3JP6UAW6 tag=KTPD5XOXKEE0 type=write size=3432490 2024-05-09T20:17:08.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.635+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~ZQCSTIL4COSH.0 size=608186 2024-05-09T20:17:08.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.635+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=ZQCSTIL4COSH tag=TGZZHWJTR2X5 type=write size=1656762 2024-05-09T20:17:08.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.639+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~QQUAP87IHRP9.0 size=1048576 2024-05-09T20:17:08.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.639+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~QQUAP87IHRP9.1 size=2358 2024-05-09T20:17:08.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.647+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=QQUAP87IHRP9 tag=ZTD8RXZ6JVPD type=write size=2099510 2024-05-09T20:17:08.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.647+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=ZEGRWPVMBU0R type=del size=1852550 2024-05-09T20:17:08.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.651+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=9U66P3P2ZS15 type=del 2024-05-09T20:17:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.655+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=7SCNP7239386 type=del size=3652018 2024-05-09T20:17:08.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.659+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~B17Z44Z4SUC2.0 size=1048576 2024-05-09T20:17:08.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.659+0000 7fdc882e58c0 1 < canceled part key=_multipart_O07UHAIJZF0R.2~B17Z44Z4SUC2.1 size=521124 2024-05-09T20:17:08.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.667+0000 7fdc882e58c0 1 < canceled multipart upload key=O07UHAIJZF0R upload=B17Z44Z4SUC2 tag=X8MSSE1GGJPE type=write size=2618276 2024-05-09T20:17:08.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.671+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=B48KGFVY4KQY type=del size=611946 2024-05-09T20:17:08.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.671+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~F7WN3CJ3Y1JB.0 size=523451 2024-05-09T20:17:08.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.679+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=F7WN3CJ3Y1JB tag=2MXQHPX8CJJ9 type=write size=1572027 2024-05-09T20:17:08.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.683+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=S9WOT88X9OQE type=del size=2104553 2024-05-09T20:17:08.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.683+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~ALUFL3SFIJOR.0 size=1048576 2024-05-09T20:17:08.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.687+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~ALUFL3SFIJOR.1 size=1048576 2024-05-09T20:17:08.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.691+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~ALUFL3SFIJOR.2 size=489267 2024-05-09T20:17:08.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.695+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=ALUFL3SFIJOR tag=79MVYHXLXQO5 type=write size=3634995 2024-05-09T20:17:08.691 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.699+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=LX10ZBNILXUE type=del size=1909625 2024-05-09T20:17:08.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.703+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~90REFP1FZRLD.0 size=1048576 2024-05-09T20:17:08.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.703+0000 7fdc882e58c0 1 < completed part key=_multipart_5UMJFJRKUPKR.2~90REFP1FZRLD.1 size=49626 2024-05-09T20:17:08.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.711+0000 7fdc882e58c0 1 < completed multipart upload key=5UMJFJRKUPKR upload=90REFP1FZRLD tag=XOAR1RZL65Q3 type=write size=2146778 2024-05-09T20:17:08.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.711+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=9RTE6YR3XK9N type=del size=145453 2024-05-09T20:17:08.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.715+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~X51FNOXTH9Y9.0 size=1048576 2024-05-09T20:17:08.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.719+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~X51FNOXTH9Y9.1 size=1048576 2024-05-09T20:17:08.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.719+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~X51FNOXTH9Y9.2 size=170124 2024-05-09T20:17:08.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.727+0000 7fdc882e58c0 1 < completed multipart upload key=HPK2T0629B1P upload=X51FNOXTH9Y9 tag=1D20W9MB0QUO type=write size=3315852 2024-05-09T20:17:08.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.727+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=NXW5BVDVA7D1 type=write size=61590 2024-05-09T20:17:08.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.731+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=2KO6D9APSCCG type=del 2024-05-09T20:17:08.729 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.735+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~3OBI5SKCP505.0 size=1048576 2024-05-09T20:17:08.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.739+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~3OBI5SKCP505.1 size=1048576 2024-05-09T20:17:08.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.743+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~3OBI5SKCP505.2 size=197376 2024-05-09T20:17:08.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.743+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=3OBI5SKCP505 tag=6Z6PVPJQQAVN type=write size=3343104 2024-05-09T20:17:08.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.747+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~N38DAW32G7WB.0 size=51815 2024-05-09T20:17:08.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.751+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=N38DAW32G7WB tag=9148MQCKZSY0 type=write size=1100391 2024-05-09T20:17:08.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.755+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=C2E2NPUVONCC type=write size=1027573 2024-05-09T20:17:08.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.759+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=YX1SQZUBW59C type=del 2024-05-09T20:17:08.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.763+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~3BC09KG76ZBU.0 size=1048576 2024-05-09T20:17:08.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.767+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~3BC09KG76ZBU.1 size=1048576 2024-05-09T20:17:08.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.767+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~3BC09KG76ZBU.2 size=436971 2024-05-09T20:17:08.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.767+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=3BC09KG76ZBU tag=8J3GN7GQBB1U type=write size=3582699 2024-05-09T20:17:08.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.775+0000 7fdc882e58c0 1 < suggested update operation key=5H2GWM9LQELZ tag=LKB7154E9LR9 type=del size=3556474 2024-05-09T20:17:08.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.779+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~J1E2G936P7JB.0 size=1048576 2024-05-09T20:17:08.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.783+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~J1E2G936P7JB.1 size=704356 2024-05-09T20:17:08.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.783+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=J1E2G936P7JB tag=OSONTQS3C111 type=write size=2801508 2024-05-09T20:17:08.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.787+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=8DQ17LBEBZ1J type=del 2024-05-09T20:17:08.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.791+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~297YAJ9TP8ZT.0 size=803974 2024-05-09T20:17:08.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.791+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=297YAJ9TP8ZT tag=FSVM7EQZ2D1N type=write size=1852550 2024-05-09T20:17:08.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.795+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=A1MCRU4H413R type=del 2024-05-09T20:17:08.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.799+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~WE457HX2GZT5.0 size=1048576 2024-05-09T20:17:08.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.803+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~WE457HX2GZT5.1 size=670023 2024-05-09T20:17:08.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.803+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=WE457HX2GZT5 tag=951RTYB8GLZN type=write size=2767175 2024-05-09T20:17:08.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.807+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~9FRTGW5XUQZJ.0 size=1048576 2024-05-09T20:17:08.801 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.807+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~9FRTGW5XUQZJ.1 size=1048576 2024-05-09T20:17:08.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.811+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~9FRTGW5XUQZJ.2 size=1024858 2024-05-09T20:17:08.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.815+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=9FRTGW5XUQZJ tag=B71N19HJ7INJ type=write size=4170586 2024-05-09T20:17:08.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.823+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~MLAPFO9FG4F1.0 size=1048576 2024-05-09T20:17:08.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.823+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~MLAPFO9FG4F1.1 size=1048576 2024-05-09T20:17:08.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.827+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~MLAPFO9FG4F1.2 size=202601 2024-05-09T20:17:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.827+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=MLAPFO9FG4F1 tag=59QUXV5205QO type=write size=3348329 2024-05-09T20:17:08.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.831+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=4AN3KFPRNC86 type=del 2024-05-09T20:17:08.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.835+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=FXMM0ZHQNG3Z type=del size=1860683 2024-05-09T20:17:08.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.835+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~ZQCSTIL4COSH.0 size=608186 2024-05-09T20:17:08.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.843+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=ZQCSTIL4COSH tag=TGZZHWJTR2X5 type=write size=1656762 2024-05-09T20:17:08.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.847+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~UD452ZGRWDAV.0 size=1048576 2024-05-09T20:17:08.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.851+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~UD452ZGRWDAV.1 size=1048576 2024-05-09T20:17:08.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.851+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~UD452ZGRWDAV.2 size=221076 2024-05-09T20:17:08.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.851+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=UD452ZGRWDAV tag=ZU9R9GY2RVVI type=write size=3366804 2024-05-09T20:17:08.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.855+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=ZEGRWPVMBU0R type=del 2024-05-09T20:17:08.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.859+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=ESHWEL22MX4V type=write size=658547 2024-05-09T20:17:08.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.863+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=7SCNP7239386 type=del 2024-05-09T20:17:08.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.867+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=NACLHNQLHSEZ type=del size=609500 2024-05-09T20:17:08.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.867+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=B48KGFVY4KQY type=del 2024-05-09T20:17:08.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.871+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=IY9DLFIBGIU9 type=del size=446817 2024-05-09T20:17:08.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.875+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=S9WOT88X9OQE type=del 2024-05-09T20:17:08.872 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.879+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=EABG5P7Z8LKF type=del size=1946812 2024-05-09T20:17:08.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.883+0000 7fdc882e58c0 1 < completed delete operation key=T6T3IOW9VPA0 tag=LX10ZBNILXUE type=del 2024-05-09T20:17:08.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.887+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=RNYCLMUP1944 type=del size=3990876 2024-05-09T20:17:08.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.891+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=9RTE6YR3XK9N type=del 2024-05-09T20:17:08.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.895+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~9UHMOHEISSAB.0 size=1048576 2024-05-09T20:17:08.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.899+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~9UHMOHEISSAB.1 size=1048576 2024-05-09T20:17:08.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.899+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~9UHMOHEISSAB.2 size=950299 2024-05-09T20:17:08.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.899+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=9UHMOHEISSAB tag=4TKAR13MWBZ8 type=write size=4096027 2024-05-09T20:17:08.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.907+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=NXW5BVDVA7D1 type=write size=61590 2024-05-09T20:17:08.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.911+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=R3QWZHY5E8BF type=del size=663265 2024-05-09T20:17:08.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.915+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~3OBI5SKCP505.0 size=1048576 2024-05-09T20:17:08.909 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.915+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~3OBI5SKCP505.1 size=1048576 2024-05-09T20:17:08.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.919+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~3OBI5SKCP505.2 size=197376 2024-05-09T20:17:08.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.923+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=3OBI5SKCP505 tag=6Z6PVPJQQAVN type=write size=3343104 2024-05-09T20:17:08.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.931+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~6JNQUDCN6LVP.0 size=1048576 2024-05-09T20:17:08.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.935+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~6JNQUDCN6LVP.1 size=891824 2024-05-09T20:17:08.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.935+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=6JNQUDCN6LVP tag=QBQP2Z5R0P0Y type=write size=2988976 2024-05-09T20:17:08.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.935+0000 7fdc882e58c0 1 < completed write operation key=8FFZZDURUWM1 tag=C2E2NPUVONCC type=write size=1027573 2024-05-09T20:17:08.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.943+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~HS944MOA9X2Y.0 size=301396 2024-05-09T20:17:08.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.943+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=HS944MOA9X2Y tag=OFA0EZ13MQTD type=write size=1349972 2024-05-09T20:17:08.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.943+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~3BC09KG76ZBU.0 size=1048576 2024-05-09T20:17:08.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.947+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~3BC09KG76ZBU.1 size=1048576 2024-05-09T20:17:08.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.947+0000 7fdc882e58c0 1 < canceled part key=_multipart_WC5YH0OU51WX.2~3BC09KG76ZBU.2 size=436971 2024-05-09T20:17:08.945 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.951+0000 7fdc882e58c0 1 < canceled multipart upload key=WC5YH0OU51WX upload=3BC09KG76ZBU tag=8J3GN7GQBB1U type=write size=3582699 2024-05-09T20:17:08.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.959+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=ACPSD0KK0JTC type=del size=2655500 2024-05-09T20:17:08.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.959+0000 7fdc882e58c0 1 < canceled part key=_multipart_EIMPFFK4I6NZ.2~J1E2G936P7JB.0 size=1048576 2024-05-09T20:17:08.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.959+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~J1E2G936P7JB.1 size=704356 2024-05-09T20:17:08.958 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.967+0000 7fdc882e58c0 1 < canceled multipart upload key=EIMPFFK4I6NZ upload=J1E2G936P7JB tag=OSONTQS3C111 type=write size=2801508 2024-05-09T20:17:08.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.971+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~EC15RA5HCCLS.0 size=137431 2024-05-09T20:17:08.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.971+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=EC15RA5HCCLS tag=I6UGZG5RFBMK type=write size=1186007 2024-05-09T20:17:08.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.975+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~297YAJ9TP8ZT.0 size=803974 2024-05-09T20:17:08.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.979+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=297YAJ9TP8ZT tag=FSVM7EQZ2D1N type=write size=1852550 2024-05-09T20:17:08.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.983+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~Q8VSIPOQEHPD.0 size=1048576 2024-05-09T20:17:08.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.987+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~Q8VSIPOQEHPD.1 size=1048576 2024-05-09T20:17:08.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.987+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~Q8VSIPOQEHPD.2 size=166990 2024-05-09T20:17:08.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.987+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=Q8VSIPOQEHPD tag=ZNNVURTV44BX type=write size=3312718 2024-05-09T20:17:08.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.991+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~WE457HX2GZT5.0 size=1048576 2024-05-09T20:17:08.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.995+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~WE457HX2GZT5.1 size=670023 2024-05-09T20:17:08.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:08.999+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=WE457HX2GZT5 tag=951RTYB8GLZN type=write size=2767175 2024-05-09T20:17:08.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.007+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~EJM64YXXPMS3.0 size=1048576 2024-05-09T20:17:09.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.007+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~EJM64YXXPMS3.1 size=1048576 2024-05-09T20:17:09.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.011+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~EJM64YXXPMS3.2 size=610946 2024-05-09T20:17:09.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.011+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=EJM64YXXPMS3 tag=439RU49JHBK6 type=write size=3756674 2024-05-09T20:17:09.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.015+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~MLAPFO9FG4F1.0 size=1048576 2024-05-09T20:17:09.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.015+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~MLAPFO9FG4F1.1 size=1048576 2024-05-09T20:17:09.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.019+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~MLAPFO9FG4F1.2 size=202601 2024-05-09T20:17:09.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.023+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=MLAPFO9FG4F1 tag=59QUXV5205QO type=write size=3348329 2024-05-09T20:17:09.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.027+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=C1P4E8OVAR4I type=write size=933519 2024-05-09T20:17:09.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.031+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=FXMM0ZHQNG3Z type=del 2024-05-09T20:17:09.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.035+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=WDADXI4U06ID type=del size=1274398 2024-05-09T20:17:09.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.039+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~UD452ZGRWDAV.0 size=1048576 2024-05-09T20:17:09.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.043+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~UD452ZGRWDAV.1 size=1048576 2024-05-09T20:17:09.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.043+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~UD452ZGRWDAV.2 size=221076 2024-05-09T20:17:09.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.047+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=UD452ZGRWDAV tag=ZU9R9GY2RVVI type=write size=3366804 2024-05-09T20:17:09.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.051+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=9MO0Z0321A4E type=del size=3489803 2024-05-09T20:17:09.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.055+0000 7fdc882e58c0 1 < completed write operation key=HPK2T0629B1P tag=ESHWEL22MX4V type=write size=658547 2024-05-09T20:17:09.052 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.059+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=Z9GG0J7JB3TJ type=del size=991603 2024-05-09T20:17:09.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.063+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=NACLHNQLHSEZ type=del 2024-05-09T20:17:09.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.067+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~38NIYY37XJ1P.0 size=960971 2024-05-09T20:17:09.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.067+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=38NIYY37XJ1P tag=S98DV8ZK1VNW type=write size=2009547 2024-05-09T20:17:09.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.071+0000 7fdc882e58c0 1 < completed delete operation key=54R9XPO62J0M tag=IY9DLFIBGIU9 type=del 2024-05-09T20:17:09.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.075+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=3C8SALPASQN1 type=del size=2041646 2024-05-09T20:17:09.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.083+0000 7fdc882e58c0 1 < suggested update operation key=TQ5ZHATLNVPP tag=EABG5P7Z8LKF type=del size=1946812 2024-05-09T20:17:09.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.091+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~1FZOTG7Q5QG1.0 size=169717 2024-05-09T20:17:09.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.091+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=1FZOTG7Q5QG1 tag=EU2534LR0R7A type=write size=1218293 2024-05-09T20:17:09.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.099+0000 7fdc882e58c0 1 < suggested remove operation key=AJANS5ADJO5B tag=RNYCLMUP1944 type=del 2024-05-09T20:17:09.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.103+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=ZUS8RKYXGBGK type=del size=1655098 2024-05-09T20:17:09.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.103+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~9UHMOHEISSAB.0 size=1048576 2024-05-09T20:17:09.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.107+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~9UHMOHEISSAB.1 size=1048576 2024-05-09T20:17:09.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.107+0000 7fdc882e58c0 1 < canceled part key=_multipart_P2DNA4DGI54P.2~9UHMOHEISSAB.2 size=950299 2024-05-09T20:17:09.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.111+0000 7fdc882e58c0 1 < canceled multipart upload key=P2DNA4DGI54P upload=9UHMOHEISSAB tag=4TKAR13MWBZ8 type=write size=4096027 2024-05-09T20:17:09.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.119+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~X8G1QIOBTRVI.0 size=1048576 2024-05-09T20:17:09.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.123+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~X8G1QIOBTRVI.1 size=746897 2024-05-09T20:17:09.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.123+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=X8G1QIOBTRVI tag=P7UWUE7Z452H type=write size=2844049 2024-05-09T20:17:09.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.127+0000 7fdc882e58c0 1 < suggested update operation key=NNGWUSCETMH4 tag=R3QWZHY5E8BF type=del size=663265 2024-05-09T20:17:09.127 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.135+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~WSW47PMC0ATB.0 size=1048576 2024-05-09T20:17:09.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.135+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~WSW47PMC0ATB.1 size=195701 2024-05-09T20:17:09.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.135+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=WSW47PMC0ATB tag=C9BJJU9YDSYU type=write size=2292853 2024-05-09T20:17:09.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.139+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~6JNQUDCN6LVP.0 size=1048576 2024-05-09T20:17:09.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.143+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~6JNQUDCN6LVP.1 size=891824 2024-05-09T20:17:09.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.147+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=6JNQUDCN6LVP tag=QBQP2Z5R0P0Y type=write size=2988976 2024-05-09T20:17:09.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.151+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=L1BGEYIKJ1Z0 type=del size=1697681 2024-05-09T20:17:09.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.155+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~HS944MOA9X2Y.0 size=301396 2024-05-09T20:17:09.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.159+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=HS944MOA9X2Y tag=OFA0EZ13MQTD type=write size=1349972 2024-05-09T20:17:09.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.163+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~LBXYLJ7L6EOE.0 size=1048576 2024-05-09T20:17:09.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.167+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~LBXYLJ7L6EOE.1 size=1048576 2024-05-09T20:17:09.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.167+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~LBXYLJ7L6EOE.2 size=800152 2024-05-09T20:17:09.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.167+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=LBXYLJ7L6EOE tag=RIO6T1I0OESV type=write size=3945880 2024-05-09T20:17:09.164 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.171+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=ACPSD0KK0JTC type=del 2024-05-09T20:17:09.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.175+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=OW8S2K8FE5U1 type=del size=237191 2024-05-09T20:17:09.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.179+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~EC15RA5HCCLS.0 size=137431 2024-05-09T20:17:09.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.183+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=EC15RA5HCCLS tag=I6UGZG5RFBMK type=write size=1186007 2024-05-09T20:17:09.179 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.187+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=0V9HDJJ6SYON type=del size=2102441 2024-05-09T20:17:09.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.191+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~Q8VSIPOQEHPD.0 size=1048576 2024-05-09T20:17:09.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.195+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~Q8VSIPOQEHPD.1 size=1048576 2024-05-09T20:17:09.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.195+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~Q8VSIPOQEHPD.2 size=166990 2024-05-09T20:17:09.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.203+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=Q8VSIPOQEHPD tag=ZNNVURTV44BX type=write size=3312718 2024-05-09T20:17:09.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.211+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~OWZ4CW5OKYKY.0 size=1048576 2024-05-09T20:17:09.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.211+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~OWZ4CW5OKYKY.1 size=1048576 2024-05-09T20:17:09.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.215+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~OWZ4CW5OKYKY.2 size=319995 2024-05-09T20:17:09.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.215+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=OWZ4CW5OKYKY tag=3IK8DYFTNOSW type=write size=3465723 2024-05-09T20:17:09.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.219+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~EJM64YXXPMS3.0 size=1048576 2024-05-09T20:17:09.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.219+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~EJM64YXXPMS3.1 size=1048576 2024-05-09T20:17:09.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.223+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~EJM64YXXPMS3.2 size=610946 2024-05-09T20:17:09.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.227+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=EJM64YXXPMS3 tag=439RU49JHBK6 type=write size=3756674 2024-05-09T20:17:09.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.235+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~6IGZFFYISG6K.0 size=967967 2024-05-09T20:17:09.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.235+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=6IGZFFYISG6K tag=V9N13DNOD5YH type=write size=2016543 2024-05-09T20:17:09.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.239+0000 7fdc882e58c0 -1 < failed to complete operation key=AJANS5ADJO5B tag=C1P4E8OVAR4I type=write size=933519: (22) Invalid argument 2024-05-09T20:17:09.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.243+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~4MOK0QICZMYL.0 size=1048576 2024-05-09T20:17:09.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.247+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~4MOK0QICZMYL.1 size=1048576 2024-05-09T20:17:09.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.247+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~4MOK0QICZMYL.2 size=492411 2024-05-09T20:17:09.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.247+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=4MOK0QICZMYL tag=7A7XRI2KZESR type=write size=3638139 2024-05-09T20:17:09.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.255+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=WDADXI4U06ID type=del 2024-05-09T20:17:09.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.259+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=3ECQCK7EBXJK type=del size=3923027 2024-05-09T20:17:09.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.267+0000 7fdc882e58c0 1 < suggested remove operation key=SNPNT5JFJOUS tag=9MO0Z0321A4E type=del 2024-05-09T20:17:09.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.271+0000 7fdc882e58c0 1 > prepared part key=_multipart_NYHR6GTPQYZS.2~L829T0UGQWXK.0 size=895915 2024-05-09T20:17:09.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.271+0000 7fdc882e58c0 1 > prepared multipart upload key=NYHR6GTPQYZS upload=L829T0UGQWXK tag=VMNMUC6JUB81 type=write size=1944491 2024-05-09T20:17:09.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.275+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=Z9GG0J7JB3TJ type=del 2024-05-09T20:17:09.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.279+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=1MZCNK8CQMB7 type=write size=749557 2024-05-09T20:17:09.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.279+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~38NIYY37XJ1P.0 size=960971 2024-05-09T20:17:09.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.287+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=38NIYY37XJ1P tag=S98DV8ZK1VNW type=write size=2009547 2024-05-09T20:17:09.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.291+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=53L7JJGL1RRH type=write size=979265 2024-05-09T20:17:09.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.295+0000 7fdc882e58c0 1 < completed delete operation key=OGVA2778MJWU tag=3C8SALPASQN1 type=del 2024-05-09T20:17:09.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.299+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~451UEU2RVJOT.0 size=587313 2024-05-09T20:17:09.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.299+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=451UEU2RVJOT tag=YOKZ0TAV0382 type=write size=1635889 2024-05-09T20:17:09.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.303+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~1FZOTG7Q5QG1.0 size=169717 2024-05-09T20:17:09.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.307+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=1FZOTG7Q5QG1 tag=EU2534LR0R7A type=write size=1218293 2024-05-09T20:17:09.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.311+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=GPQDGAVDTVCK type=del size=1858414 2024-05-09T20:17:09.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.315+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=ZUS8RKYXGBGK type=del 2024-05-09T20:17:09.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.323+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~SQ2N2TI9HLYP.0 size=1048576 2024-05-09T20:17:09.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.323+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~SQ2N2TI9HLYP.1 size=762205 2024-05-09T20:17:09.318 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.323+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=SQ2N2TI9HLYP tag=HW0M3DEWTK3A type=write size=2859357 2024-05-09T20:17:09.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.327+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~X8G1QIOBTRVI.0 size=1048576 2024-05-09T20:17:09.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.331+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~X8G1QIOBTRVI.1 size=746897 2024-05-09T20:17:09.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.335+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=X8G1QIOBTRVI tag=P7UWUE7Z452H type=write size=2844049 2024-05-09T20:17:09.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.343+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~2BH746GYPA7W.0 size=954225 2024-05-09T20:17:09.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.343+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=2BH746GYPA7W tag=EN9WQTBGT75S type=write size=2002801 2024-05-09T20:17:09.339 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.347+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~WSW47PMC0ATB.0 size=1048576 2024-05-09T20:17:09.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.351+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~WSW47PMC0ATB.1 size=195701 2024-05-09T20:17:09.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.355+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=WSW47PMC0ATB tag=C9BJJU9YDSYU type=write size=2292853 2024-05-09T20:17:09.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.359+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=HVBVD3RFVNTD type=del size=2452345 2024-05-09T20:17:09.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.363+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=L1BGEYIKJ1Z0 type=del 2024-05-09T20:17:09.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.363+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=200DL1XOUIRZ type=write size=160314 2024-05-09T20:17:09.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.367+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~LBXYLJ7L6EOE.0 size=1048576 2024-05-09T20:17:09.364 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.371+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~LBXYLJ7L6EOE.1 size=1048576 2024-05-09T20:17:09.367 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.375+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~LBXYLJ7L6EOE.2 size=800152 2024-05-09T20:17:09.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.379+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=LBXYLJ7L6EOE tag=RIO6T1I0OESV type=write size=3945880 2024-05-09T20:17:09.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.383+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=MKDUQ7J7ESTD type=del size=3907811 2024-05-09T20:17:09.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.387+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=OW8S2K8FE5U1 type=del 2024-05-09T20:17:09.383 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.391+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=O4WRPG1SWL3I type=del size=2687520 2024-05-09T20:17:09.384 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.391+0000 7fdc882e58c0 -1 < no bi entry to suggest for operation key=SNPNT5JFJOUS tag=0V9HDJJ6SYON type=del size=2102441: (2) No such file or directory 2024-05-09T20:17:09.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.395+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=612Z2TP713XJ type=write size=387301 2024-05-09T20:17:09.391 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.399+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~OWZ4CW5OKYKY.0 size=1048576 2024-05-09T20:17:09.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.403+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~OWZ4CW5OKYKY.1 size=1048576 2024-05-09T20:17:09.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.403+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~OWZ4CW5OKYKY.2 size=319995 2024-05-09T20:17:09.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.407+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=OWZ4CW5OKYKY tag=3IK8DYFTNOSW type=write size=3465723 2024-05-09T20:17:09.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.415+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~L96VI9Z9IO9Q.0 size=1048576 2024-05-09T20:17:09.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.415+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~L96VI9Z9IO9Q.1 size=312220 2024-05-09T20:17:09.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.415+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=L96VI9Z9IO9Q tag=1E3DSTVQ41CB type=write size=2409372 2024-05-09T20:17:09.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.419+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~6IGZFFYISG6K.0 size=967967 2024-05-09T20:17:09.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.423+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=6IGZFFYISG6K tag=V9N13DNOD5YH type=write size=2016543 2024-05-09T20:17:09.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.427+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=E6048E1OS80H type=write size=129442 2024-05-09T20:17:09.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.431+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~4MOK0QICZMYL.0 size=1048576 2024-05-09T20:17:09.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.431+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~4MOK0QICZMYL.1 size=1048576 2024-05-09T20:17:09.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.435+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~4MOK0QICZMYL.2 size=492411 2024-05-09T20:17:09.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.439+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=4MOK0QICZMYL tag=7A7XRI2KZESR type=write size=3638139 2024-05-09T20:17:09.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.443+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=X28AMQJX02RK type=del size=4029424 2024-05-09T20:17:09.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.447+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=3ECQCK7EBXJK type=del 2024-05-09T20:17:09.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.447+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=RBN5H1XYU9WG type=del size=1811644 2024-05-09T20:17:09.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.451+0000 7fdc882e58c0 1 < completed part key=_multipart_NYHR6GTPQYZS.2~L829T0UGQWXK.0 size=895915 2024-05-09T20:17:09.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.455+0000 7fdc882e58c0 1 < completed multipart upload key=NYHR6GTPQYZS upload=L829T0UGQWXK tag=VMNMUC6JUB81 type=write size=1944491 2024-05-09T20:17:09.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.463+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~WHAKRQC9ZM5O.0 size=390725 2024-05-09T20:17:09.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.463+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=WHAKRQC9ZM5O tag=F624BZ8EDIUX type=write size=1439301 2024-05-09T20:17:09.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.467+0000 7fdc882e58c0 1 < suggested update operation key=O07UHAIJZF0R tag=1MZCNK8CQMB7 type=write size=749557 2024-05-09T20:17:09.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.475+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~2AASDET3JU3F.0 size=1048576 2024-05-09T20:17:09.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.475+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~2AASDET3JU3F.1 size=169162 2024-05-09T20:17:09.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.475+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=2AASDET3JU3F tag=OWBIE24Y62GC type=write size=2266314 2024-05-09T20:17:09.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.479+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=53L7JJGL1RRH type=write size=979265 2024-05-09T20:17:09.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.483+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=L3SPLEMJJY7P type=del size=1306225 2024-05-09T20:17:09.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.487+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~451UEU2RVJOT.0 size=587313 2024-05-09T20:17:09.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.491+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=451UEU2RVJOT tag=YOKZ0TAV0382 type=write size=1635889 2024-05-09T20:17:09.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.495+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=KUOBP0PWRJV2 type=del size=186300 2024-05-09T20:17:09.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.495+0000 7fdc882e58c0 1 < canceled operation key=HPK2T0629B1P tag=GPQDGAVDTVCK type=del size=1858414 2024-05-09T20:17:09.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.499+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=U9RCQRQ7CX5G type=del size=2743445 2024-05-09T20:17:09.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.503+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~SQ2N2TI9HLYP.0 size=1048576 2024-05-09T20:17:09.497 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.503+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~SQ2N2TI9HLYP.1 size=762205 2024-05-09T20:17:09.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.511+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=SQ2N2TI9HLYP tag=HW0M3DEWTK3A type=write size=2859357 2024-05-09T20:17:09.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.511+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=00SJQGQSWAD6 type=del size=4057452 2024-05-09T20:17:09.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.515+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~2BH746GYPA7W.0 size=954225 2024-05-09T20:17:09.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.519+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=2BH746GYPA7W tag=EN9WQTBGT75S type=write size=2002801 2024-05-09T20:17:09.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.523+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=8X3KTX0E8QYU type=del size=2138171 2024-05-09T20:17:09.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.527+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=HVBVD3RFVNTD type=del size=2452345 2024-05-09T20:17:09.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.531+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=W64S7M132EEZ type=del size=3661281 2024-05-09T20:17:09.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.535+0000 7fdc882e58c0 1 < completed write operation key=J2Q86HKRMAKE tag=200DL1XOUIRZ type=write size=160314 2024-05-09T20:17:09.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.539+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=L6R2SZVNYSID type=write size=1017120 2024-05-09T20:17:09.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.539+0000 7fdc882e58c0 1 < completed delete operation key=FNB9B18LIA8L tag=MKDUQ7J7ESTD type=del 2024-05-09T20:17:09.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.543+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=0AO24UXEHUEC type=write size=161062 2024-05-09T20:17:09.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.555+0000 7fdc882e58c0 1 < suggested update operation key=WC5YH0OU51WX tag=O4WRPG1SWL3I type=del size=2687520 2024-05-09T20:17:09.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.555+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=SH7KZR2DG7Y0 type=del size=4021610 2024-05-09T20:17:09.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.559+0000 7fdc882e58c0 1 < completed write operation key=JS90W6PIREMD tag=612Z2TP713XJ type=write size=387301 2024-05-09T20:17:09.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.563+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~GU005Y9SS6UG.0 size=1048576 2024-05-09T20:17:09.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.567+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~GU005Y9SS6UG.1 size=1048576 2024-05-09T20:17:09.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.567+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~GU005Y9SS6UG.2 size=692987 2024-05-09T20:17:09.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.567+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=GU005Y9SS6UG tag=UNIIWMQJZ6W6 type=write size=3838715 2024-05-09T20:17:09.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.571+0000 7fdc882e58c0 1 < canceled part key=_multipart_T6T3IOW9VPA0.2~L96VI9Z9IO9Q.0 size=1048576 2024-05-09T20:17:09.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.571+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~L96VI9Z9IO9Q.1 size=312220 2024-05-09T20:17:09.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.575+0000 7fdc882e58c0 1 < canceled multipart upload key=T6T3IOW9VPA0 upload=L96VI9Z9IO9Q tag=1E3DSTVQ41CB type=write size=2409372 2024-05-09T20:17:09.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.579+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~L0LVH9HXSQH2.0 size=1048576 2024-05-09T20:17:09.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.583+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~L0LVH9HXSQH2.1 size=567686 2024-05-09T20:17:09.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.583+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=L0LVH9HXSQH2 tag=CG7GP362D6QI type=write size=2664838 2024-05-09T20:17:09.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.587+0000 7fdc882e58c0 1 < completed write operation key=TWWOL5D8C1MQ tag=E6048E1OS80H type=write size=129442 2024-05-09T20:17:09.581 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.587+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=MDBFSL1DM8V3 type=del size=1396097 2024-05-09T20:17:09.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.591+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=X28AMQJX02RK type=del 2024-05-09T20:17:09.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.595+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~NHJC4J5XS9WN.0 size=612514 2024-05-09T20:17:09.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.595+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=NHJC4J5XS9WN tag=22A2A58ZGCAP type=write size=1661090 2024-05-09T20:17:09.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.603+0000 7fdc882e58c0 1 < suggested update operation key=G0DX1F257RLD tag=RBN5H1XYU9WG type=del size=1811644 2024-05-09T20:17:09.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.603+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=1CVNIW0Q8RRY type=del size=2136793 2024-05-09T20:17:09.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.607+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~WHAKRQC9ZM5O.0 size=390725 2024-05-09T20:17:09.603 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.611+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=WHAKRQC9ZM5O tag=F624BZ8EDIUX type=write size=1439301 2024-05-09T20:17:09.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.615+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~W56FIHMLMLZ2.0 size=1048576 2024-05-09T20:17:09.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.615+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~W56FIHMLMLZ2.1 size=1048576 2024-05-09T20:17:09.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.619+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~W56FIHMLMLZ2.2 size=128495 2024-05-09T20:17:09.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.619+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=W56FIHMLMLZ2 tag=847MAVEWN5Y9 type=write size=3274223 2024-05-09T20:17:09.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.623+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~2AASDET3JU3F.0 size=1048576 2024-05-09T20:17:09.616 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.623+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~2AASDET3JU3F.1 size=169162 2024-05-09T20:17:09.621 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.627+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=2AASDET3JU3F tag=OWBIE24Y62GC type=write size=2266314 2024-05-09T20:17:09.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.635+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~EMY9R70EOP1Q.0 size=1048576 2024-05-09T20:17:09.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.635+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~EMY9R70EOP1Q.1 size=1048576 2024-05-09T20:17:09.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.639+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~EMY9R70EOP1Q.2 size=123684 2024-05-09T20:17:09.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.639+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=EMY9R70EOP1Q tag=M53P929K8VR9 type=write size=3269412 2024-05-09T20:17:09.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.643+0000 7fdc882e58c0 1 < suggested remove operation key=SNPNT5JFJOUS tag=L3SPLEMJJY7P type=del 2024-05-09T20:17:09.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.647+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=HE00ZJ0WAFB1 type=del size=2885972 2024-05-09T20:17:09.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.647+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=KUOBP0PWRJV2 type=del 2024-05-09T20:17:09.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.651+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=LNZ0UNT6BT4K type=del size=3174116 2024-05-09T20:17:09.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.659+0000 7fdc882e58c0 1 < suggested update operation key=8FFZZDURUWM1 tag=U9RCQRQ7CX5G type=del size=2743445 2024-05-09T20:17:09.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.659+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=JPSZO347ULX3 type=write size=463170 2024-05-09T20:17:09.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.663+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=00SJQGQSWAD6 type=del 2024-05-09T20:17:09.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.667+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=TVN4NCRD1VEM type=del size=590347 2024-05-09T20:17:09.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.671+0000 7fdc882e58c0 -1 < failed to complete operation key=EK78CNKUHNKO tag=8X3KTX0E8QYU type=del size=2138171: (22) Invalid argument 2024-05-09T20:17:09.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.675+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=DTPJ4KF575JS type=del size=1080533 2024-05-09T20:17:09.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.675+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=W64S7M132EEZ type=del 2024-05-09T20:17:09.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.683+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~OA4L68A1FA51.0 size=1048576 2024-05-09T20:17:09.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.687+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~OA4L68A1FA51.1 size=1048576 2024-05-09T20:17:09.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.687+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~OA4L68A1FA51.2 size=626022 2024-05-09T20:17:09.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.687+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=OA4L68A1FA51 tag=E91NDAOPOBEA type=write size=3771750 2024-05-09T20:17:09.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.695+0000 7fdc882e58c0 1 < suggested remove operation key=HPK2T0629B1P tag=L6R2SZVNYSID type=write 2024-05-09T20:17:09.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.699+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=DECMBTFO74CP type=del size=1545857 2024-05-09T20:17:09.695 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.703+0000 7fdc882e58c0 1 < completed write operation key=QM53IENH9CX4 tag=0AO24UXEHUEC type=write size=161062 2024-05-09T20:17:09.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.703+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=4F7NC9A709AR type=del size=82201 2024-05-09T20:17:09.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.707+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=SH7KZR2DG7Y0 type=del 2024-05-09T20:17:09.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.711+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=VWO5H6JGIB0W type=del size=2878722 2024-05-09T20:17:09.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.715+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~GU005Y9SS6UG.0 size=1048576 2024-05-09T20:17:09.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.715+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~GU005Y9SS6UG.1 size=1048576 2024-05-09T20:17:09.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.719+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~GU005Y9SS6UG.2 size=692987 2024-05-09T20:17:09.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.723+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=GU005Y9SS6UG tag=UNIIWMQJZ6W6 type=write size=3838715 2024-05-09T20:17:09.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.727+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=W9TYUNI4K3Y2 type=write size=156256 2024-05-09T20:17:09.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.727+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~L0LVH9HXSQH2.0 size=1048576 2024-05-09T20:17:09.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.731+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~L0LVH9HXSQH2.1 size=567686 2024-05-09T20:17:09.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.739+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=L0LVH9HXSQH2 tag=CG7GP362D6QI type=write size=2664838 2024-05-09T20:17:09.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.743+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=SK4EXQWS8X15 type=del size=71174 2024-05-09T20:17:09.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.747+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=MDBFSL1DM8V3 type=del 2024-05-09T20:17:09.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.751+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=ZI9UTZ59LE25 type=del size=2447126 2024-05-09T20:17:09.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.751+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~NHJC4J5XS9WN.0 size=612514 2024-05-09T20:17:09.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.759+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=NHJC4J5XS9WN tag=22A2A58ZGCAP type=write size=1661090 2024-05-09T20:17:09.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.763+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~DM4001UQX7Z5.0 size=1048576 2024-05-09T20:17:09.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.767+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~DM4001UQX7Z5.1 size=1048576 2024-05-09T20:17:09.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.767+0000 7fdc882e58c0 1 > prepared part key=_multipart_IZPPFLWVPRKO.2~DM4001UQX7Z5.2 size=1002358 2024-05-09T20:17:09.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.767+0000 7fdc882e58c0 1 > prepared multipart upload key=IZPPFLWVPRKO upload=DM4001UQX7Z5 tag=QZOOC48O6XAE type=write size=4148086 2024-05-09T20:17:09.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.771+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=1CVNIW0Q8RRY type=del 2024-05-09T20:17:09.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.779+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~JZGR90FAH3L7.0 size=363754 2024-05-09T20:17:09.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.779+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=JZGR90FAH3L7 tag=914N4BIHAK37 type=write size=1412330 2024-05-09T20:17:09.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.783+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~W56FIHMLMLZ2.0 size=1048576 2024-05-09T20:17:09.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.783+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~W56FIHMLMLZ2.1 size=1048576 2024-05-09T20:17:09.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.787+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~W56FIHMLMLZ2.2 size=128495 2024-05-09T20:17:09.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.795+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=W56FIHMLMLZ2 tag=847MAVEWN5Y9 type=write size=3274223 2024-05-09T20:17:09.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.799+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=T2TX5WIBGC6W type=del size=3423609 2024-05-09T20:17:09.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.799+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~EMY9R70EOP1Q.0 size=1048576 2024-05-09T20:17:09.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.803+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~EMY9R70EOP1Q.1 size=1048576 2024-05-09T20:17:09.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.807+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~EMY9R70EOP1Q.2 size=123684 2024-05-09T20:17:09.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.811+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=EMY9R70EOP1Q tag=M53P929K8VR9 type=write size=3269412 2024-05-09T20:17:09.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.819+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~GHV4G36L5UWG.0 size=593750 2024-05-09T20:17:09.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.819+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=GHV4G36L5UWG tag=Y9GIUTHFVPCY type=write size=1642326 2024-05-09T20:17:09.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.827+0000 7fdc882e58c0 1 < suggested update operation key=NYHR6GTPQYZS tag=HE00ZJ0WAFB1 type=del size=2885972 2024-05-09T20:17:09.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.831+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=E3MF3OSKFU2H type=del size=1347675 2024-05-09T20:17:09.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.835+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=LNZ0UNT6BT4K type=del 2024-05-09T20:17:09.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.835+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=5DTS71DAN5FO type=write size=439499 2024-05-09T20:17:09.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.843+0000 7fdc882e58c0 1 < completed write operation key=5H2GWM9LQELZ tag=JPSZO347ULX3 type=write size=463170 2024-05-09T20:17:09.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.847+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~FUNDMWFUUBPX.0 size=1048576 2024-05-09T20:17:09.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.851+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~FUNDMWFUUBPX.1 size=434625 2024-05-09T20:17:09.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.851+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=FUNDMWFUUBPX tag=CIUKG4W6RJML type=write size=2531777 2024-05-09T20:17:09.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.859+0000 7fdc882e58c0 1 < suggested remove operation key=BS0T1OQOBS67 tag=TVN4NCRD1VEM type=del 2024-05-09T20:17:09.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.863+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~J3GDDU699NCP.0 size=1048576 2024-05-09T20:17:09.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.867+0000 7fdc882e58c0 1 > prepared part key=_multipart_JS90W6PIREMD.2~J3GDDU699NCP.1 size=1024166 2024-05-09T20:17:09.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.867+0000 7fdc882e58c0 1 > prepared multipart upload key=JS90W6PIREMD upload=J3GDDU699NCP tag=RAXGZAGGW0G4 type=write size=3121318 2024-05-09T20:17:09.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.871+0000 7fdc882e58c0 1 < canceled operation key=J2Q86HKRMAKE tag=DTPJ4KF575JS type=del size=1080533 2024-05-09T20:17:09.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.871+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=YL42B4KSL3XX type=del size=3427679 2024-05-09T20:17:09.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.875+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~OA4L68A1FA51.0 size=1048576 2024-05-09T20:17:09.870 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.879+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~OA4L68A1FA51.1 size=1048576 2024-05-09T20:17:09.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.879+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~OA4L68A1FA51.2 size=626022 2024-05-09T20:17:09.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.887+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=OA4L68A1FA51 tag=E91NDAOPOBEA type=write size=3771750 2024-05-09T20:17:09.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.891+0000 7fdc882e58c0 1 > prepared part key=_multipart_BS0T1OQOBS67.2~HNH3I1P7D5JY.0 size=647401 2024-05-09T20:17:09.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.891+0000 7fdc882e58c0 1 > prepared multipart upload key=BS0T1OQOBS67 upload=HNH3I1P7D5JY tag=5LVE0UBM70HR type=write size=1695977 2024-05-09T20:17:09.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.895+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=DECMBTFO74CP type=del 2024-05-09T20:17:09.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.899+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=QRVF7J7FG0VV type=del size=3659445 2024-05-09T20:17:09.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.907+0000 7fdc882e58c0 1 < suggested update operation key=AJANS5ADJO5B tag=4F7NC9A709AR type=del size=82201 2024-05-09T20:17:09.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.911+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=2ZYZ7FHATVPI type=del size=3873105 2024-05-09T20:17:09.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.915+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=VWO5H6JGIB0W type=del 2024-05-09T20:17:09.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.915+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=V7HIEM8QVDTR type=del size=3325330 2024-05-09T20:17:09.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.919+0000 7fdc882e58c0 1 < completed write operation key=VZ7XOAT899GU tag=W9TYUNI4K3Y2 type=write size=156256 2024-05-09T20:17:09.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.927+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~MTSEFD54A7NH.0 size=1048576 2024-05-09T20:17:09.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.931+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~MTSEFD54A7NH.1 size=216638 2024-05-09T20:17:09.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.931+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=MTSEFD54A7NH tag=I4VU9M7IZDJM type=write size=2313790 2024-05-09T20:17:09.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.935+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=SK4EXQWS8X15 type=del 2024-05-09T20:17:09.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.939+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=WXWF6BFQPJDH type=del size=161215 2024-05-09T20:17:09.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.943+0000 7fdc882e58c0 1 < completed delete operation key=EK78CNKUHNKO tag=ZI9UTZ59LE25 type=del 2024-05-09T20:17:09.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.947+0000 7fdc882e58c0 1 > prepared part key=_multipart_EK78CNKUHNKO.2~JALCTIT0XDYV.0 size=587040 2024-05-09T20:17:09.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.947+0000 7fdc882e58c0 1 > prepared multipart upload key=EK78CNKUHNKO upload=JALCTIT0XDYV tag=ZNTFM8IH15H3 type=write size=1635616 2024-05-09T20:17:09.945 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.951+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~DM4001UQX7Z5.0 size=1048576 2024-05-09T20:17:09.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.955+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~DM4001UQX7Z5.1 size=1048576 2024-05-09T20:17:09.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.959+0000 7fdc882e58c0 1 < completed part key=_multipart_IZPPFLWVPRKO.2~DM4001UQX7Z5.2 size=1002358 2024-05-09T20:17:09.958 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.963+0000 7fdc882e58c0 1 < completed multipart upload key=IZPPFLWVPRKO upload=DM4001UQX7Z5 tag=QZOOC48O6XAE type=write size=4148086 2024-05-09T20:17:09.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.971+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~E5QZDSQQIMQM.0 size=716290 2024-05-09T20:17:09.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.971+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=E5QZDSQQIMQM tag=S1TZNKXRZFKN type=write size=1764866 2024-05-09T20:17:09.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.975+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~JZGR90FAH3L7.0 size=363754 2024-05-09T20:17:09.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.979+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=JZGR90FAH3L7 tag=914N4BIHAK37 type=write size=1412330 2024-05-09T20:17:09.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.983+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=70OZHKN9E2QT type=del size=2227468 2024-05-09T20:17:09.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.991+0000 7fdc882e58c0 1 < suggested remove operation key=T5EL7X0P450L tag=T2TX5WIBGC6W type=del 2024-05-09T20:17:09.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.995+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=QDBEG2KZVSL0 type=del size=677695 2024-05-09T20:17:09.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:09.999+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~GHV4G36L5UWG.0 size=593750 2024-05-09T20:17:09.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.003+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=GHV4G36L5UWG tag=Y9GIUTHFVPCY type=write size=1642326 2024-05-09T20:17:10.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.007+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~92OU9WT616S5.0 size=1048576 2024-05-09T20:17:10.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.011+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~92OU9WT616S5.1 size=1048576 2024-05-09T20:17:10.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.011+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~92OU9WT616S5.2 size=865461 2024-05-09T20:17:10.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.011+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=92OU9WT616S5 tag=BCZZTXRBGRNI type=write size=4011189 2024-05-09T20:17:10.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.015+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=E3MF3OSKFU2H type=del 2024-05-09T20:17:10.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.015+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=2JROKRBGYQF2 type=del size=3029644 2024-05-09T20:17:10.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.019+0000 7fdc882e58c0 1 < completed write operation key=T6T3IOW9VPA0 tag=5DTS71DAN5FO type=write size=439499 2024-05-09T20:17:10.014 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.023+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=5UMLL0499OOB type=del size=2242157 2024-05-09T20:17:10.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.023+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~FUNDMWFUUBPX.0 size=1048576 2024-05-09T20:17:10.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.023+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~FUNDMWFUUBPX.1 size=434625 2024-05-09T20:17:10.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.027+0000 7fdc882e58c0 1 < completed multipart upload key=54R9XPO62J0M upload=FUNDMWFUUBPX tag=CIUKG4W6RJML type=write size=2531777 2024-05-09T20:17:10.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.031+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~HKC9K6BLN815.0 size=260541 2024-05-09T20:17:10.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.031+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=HKC9K6BLN815 tag=KF02IKBW1O11 type=write size=1309117 2024-05-09T20:17:10.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.035+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~J3GDDU699NCP.0 size=1048576 2024-05-09T20:17:10.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.035+0000 7fdc882e58c0 1 < completed part key=_multipart_JS90W6PIREMD.2~J3GDDU699NCP.1 size=1024166 2024-05-09T20:17:10.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.039+0000 7fdc882e58c0 1 < completed multipart upload key=JS90W6PIREMD upload=J3GDDU699NCP tag=RAXGZAGGW0G4 type=write size=3121318 2024-05-09T20:17:10.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.043+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~C4S6P2TIYVNZ.0 size=1048576 2024-05-09T20:17:10.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.047+0000 7fdc882e58c0 1 > prepared part key=_multipart_EIMPFFK4I6NZ.2~C4S6P2TIYVNZ.1 size=276904 2024-05-09T20:17:10.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.047+0000 7fdc882e58c0 1 > prepared multipart upload key=EIMPFFK4I6NZ upload=C4S6P2TIYVNZ tag=I7GMJ32P3H2X type=write size=2374056 2024-05-09T20:17:10.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.051+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=YL42B4KSL3XX type=del 2024-05-09T20:17:10.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.051+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=2B475AXJR33C type=del size=3651797 2024-05-09T20:17:10.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.055+0000 7fdc882e58c0 1 < completed part key=_multipart_BS0T1OQOBS67.2~HNH3I1P7D5JY.0 size=647401 2024-05-09T20:17:10.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.059+0000 7fdc882e58c0 1 < completed multipart upload key=BS0T1OQOBS67 upload=HNH3I1P7D5JY tag=5LVE0UBM70HR type=write size=1695977 2024-05-09T20:17:10.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.063+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~EXCQFAG72VJZ.0 size=1048576 2024-05-09T20:17:10.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.067+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~EXCQFAG72VJZ.1 size=696299 2024-05-09T20:17:10.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.067+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=EXCQFAG72VJZ tag=N3WC6NLLRE93 type=write size=2793451 2024-05-09T20:17:10.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.067+0000 7fdc882e58c0 -1 < failed to complete operation key=T5EL7X0P450L tag=QRVF7J7FG0VV type=del size=3659445: (22) Invalid argument 2024-05-09T20:17:10.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.071+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=KCON8UT0P989 type=del size=658178 2024-05-09T20:17:10.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.075+0000 7fdc882e58c0 1 < suggested remove operation key=5UMJFJRKUPKR tag=2ZYZ7FHATVPI type=del 2024-05-09T20:17:10.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.079+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~C3C2BMF7HS7M.0 size=1048576 2024-05-09T20:17:10.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.083+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~C3C2BMF7HS7M.1 size=1048576 2024-05-09T20:17:10.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.083+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~C3C2BMF7HS7M.2 size=585884 2024-05-09T20:17:10.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.083+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=C3C2BMF7HS7M tag=O2G1RZD5MQ3A type=write size=3731612 2024-05-09T20:17:10.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.087+0000 7fdc882e58c0 -1 < failed to complete operation key=5UMJFJRKUPKR tag=V7HIEM8QVDTR type=del size=3325330: (22) Invalid argument 2024-05-09T20:17:10.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.087+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=64YH4UOZY7S4 type=del size=893067 2024-05-09T20:17:10.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.091+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~MTSEFD54A7NH.0 size=1048576 2024-05-09T20:17:10.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.095+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~MTSEFD54A7NH.1 size=216638 2024-05-09T20:17:10.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.099+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=MTSEFD54A7NH tag=I4VU9M7IZDJM type=write size=2313790 2024-05-09T20:17:10.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.099+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=3P0C1JVTVPWQ type=del size=2116245 2024-05-09T20:17:10.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.103+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=WXWF6BFQPJDH type=del 2024-05-09T20:17:10.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.107+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=JY9J0VME9NS4 type=del size=171353 2024-05-09T20:17:10.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.107+0000 7fdc882e58c0 1 < completed part key=_multipart_EK78CNKUHNKO.2~JALCTIT0XDYV.0 size=587040 2024-05-09T20:17:10.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.111+0000 7fdc882e58c0 1 < completed multipart upload key=EK78CNKUHNKO upload=JALCTIT0XDYV tag=ZNTFM8IH15H3 type=write size=1635616 2024-05-09T20:17:10.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.119+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=7USHG5THGY3G type=del size=2488049 2024-05-09T20:17:10.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.123+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~E5QZDSQQIMQM.0 size=716290 2024-05-09T20:17:10.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.127+0000 7fdc882e58c0 1 < completed multipart upload key=J2Q86HKRMAKE upload=E5QZDSQQIMQM tag=S1TZNKXRZFKN type=write size=1764866 2024-05-09T20:17:10.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.131+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=WB1ZTLZBZHMY type=del size=3390588 2024-05-09T20:17:10.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.131+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=70OZHKN9E2QT type=del 2024-05-09T20:17:10.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.135+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=51RU7Z8850Z9 type=del size=1523410 2024-05-09T20:17:10.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.139+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=QDBEG2KZVSL0 type=del 2024-05-09T20:17:10.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.143+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=F3D9PRZJGVLK type=del size=811944 2024-05-09T20:17:10.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.147+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~92OU9WT616S5.0 size=1048576 2024-05-09T20:17:10.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.147+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~92OU9WT616S5.1 size=1048576 2024-05-09T20:17:10.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.151+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~92OU9WT616S5.2 size=865461 2024-05-09T20:17:10.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.155+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=92OU9WT616S5 tag=BCZZTXRBGRNI type=write size=4011189 2024-05-09T20:17:10.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.159+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=3B4CAO22JO4X type=write size=637248 2024-05-09T20:17:10.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.163+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=2JROKRBGYQF2 type=del 2024-05-09T20:17:10.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.163+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=S4Y6E9XVK61R type=del size=2042686 2024-05-09T20:17:10.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.171+0000 7fdc882e58c0 1 < suggested remove operation key=JS90W6PIREMD tag=5UMLL0499OOB type=del 2024-05-09T20:17:10.164 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.171+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=WC8N0QSVRLZP type=del size=599007 2024-05-09T20:17:10.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.175+0000 7fdc882e58c0 1 < completed part key=_multipart_O07UHAIJZF0R.2~HKC9K6BLN815.0 size=260541 2024-05-09T20:17:10.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.179+0000 7fdc882e58c0 1 < completed multipart upload key=O07UHAIJZF0R upload=HKC9K6BLN815 tag=KF02IKBW1O11 type=write size=1309117 2024-05-09T20:17:10.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.183+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~PI9Q9MGWB86P.0 size=601608 2024-05-09T20:17:10.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.183+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=PI9Q9MGWB86P tag=D0K0WEKVEWV5 type=write size=1650184 2024-05-09T20:17:10.179 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.187+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~C4S6P2TIYVNZ.0 size=1048576 2024-05-09T20:17:10.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.187+0000 7fdc882e58c0 1 < completed part key=_multipart_EIMPFFK4I6NZ.2~C4S6P2TIYVNZ.1 size=276904 2024-05-09T20:17:10.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.195+0000 7fdc882e58c0 1 < completed multipart upload key=EIMPFFK4I6NZ upload=C4S6P2TIYVNZ tag=I7GMJ32P3H2X type=write size=2374056 2024-05-09T20:17:10.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.195+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=O6JEM6VFK4UT type=del size=1049288 2024-05-09T20:17:10.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.199+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=2B475AXJR33C type=del 2024-05-09T20:17:10.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.203+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=MD57NH3ATOPJ type=write size=395695 2024-05-09T20:17:10.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.207+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~EXCQFAG72VJZ.0 size=1048576 2024-05-09T20:17:10.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.211+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~EXCQFAG72VJZ.1 size=696299 2024-05-09T20:17:10.208 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.215+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=EXCQFAG72VJZ tag=N3WC6NLLRE93 type=write size=2793451 2024-05-09T20:17:10.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.219+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=OC5REMVUQTVV type=del size=1425518 2024-05-09T20:17:10.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.223+0000 7fdc882e58c0 1 < canceled operation key=SPXHCWZICCWY tag=KCON8UT0P989 type=del size=658178 2024-05-09T20:17:10.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.227+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~LYO1CVQJI78C.0 size=1048576 2024-05-09T20:17:10.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.231+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~LYO1CVQJI78C.1 size=1048576 2024-05-09T20:17:10.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.231+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~LYO1CVQJI78C.2 size=643132 2024-05-09T20:17:10.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.231+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=LYO1CVQJI78C tag=2VHWT0Q5L7JZ type=write size=3788860 2024-05-09T20:17:10.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.235+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~C3C2BMF7HS7M.0 size=1048576 2024-05-09T20:17:10.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.239+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~C3C2BMF7HS7M.1 size=1048576 2024-05-09T20:17:10.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.239+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~C3C2BMF7HS7M.2 size=585884 2024-05-09T20:17:10.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.247+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=C3C2BMF7HS7M tag=O2G1RZD5MQ3A type=write size=3731612 2024-05-09T20:17:10.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.251+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=6GSKSHQUAJ1P type=del size=2097389 2024-05-09T20:17:10.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.255+0000 7fdc882e58c0 1 < canceled operation key=FNB9B18LIA8L tag=64YH4UOZY7S4 type=del size=893067 2024-05-09T20:17:10.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.259+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~RQ9EXA199EI9.0 size=1048576 2024-05-09T20:17:10.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.263+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~RQ9EXA199EI9.1 size=1048576 2024-05-09T20:17:10.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.263+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~RQ9EXA199EI9.2 size=282717 2024-05-09T20:17:10.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.263+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=RQ9EXA199EI9 tag=UT1NOAQKRTF0 type=write size=3428445 2024-05-09T20:17:10.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.271+0000 7fdc882e58c0 1 < suggested remove operation key=5UMJFJRKUPKR tag=3P0C1JVTVPWQ type=del 2024-05-09T20:17:10.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.275+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=UG9FIYAU5DH4 type=write size=32350 2024-05-09T20:17:10.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.279+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=JY9J0VME9NS4 type=del 2024-05-09T20:17:10.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.279+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=8B43BLVEC6ZW type=del size=3056272 2024-05-09T20:17:10.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.283+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=7USHG5THGY3G type=del 2024-05-09T20:17:10.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.287+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=3EKYCK0Q4XJY type=del size=438605 2024-05-09T20:17:10.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.295+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=WB1ZTLZBZHMY type=del size=3390588 2024-05-09T20:17:10.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.299+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=OPWZGNM52FJ6 type=del size=2483668 2024-05-09T20:17:10.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.303+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=51RU7Z8850Z9 type=del 2024-05-09T20:17:10.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.311+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=KLDYANNEOIQW type=del size=1567157 2024-05-09T20:17:10.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.315+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=F3D9PRZJGVLK type=del 2024-05-09T20:17:10.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.319+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=M1G5GOQHYGKT type=del size=2485783 2024-05-09T20:17:10.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.319+0000 7fdc882e58c0 1 < completed write operation key=FNB9B18LIA8L tag=3B4CAO22JO4X type=write size=637248 2024-05-09T20:17:10.320 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.327+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~8K4TXA1J4QZT.0 size=1048576 2024-05-09T20:17:10.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.331+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~8K4TXA1J4QZT.1 size=999317 2024-05-09T20:17:10.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.331+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=8K4TXA1J4QZT tag=G71SO5EFSNS1 type=write size=3096469 2024-05-09T20:17:10.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.335+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=S4Y6E9XVK61R type=del 2024-05-09T20:17:10.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.339+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~Z8RDL1NNGTPG.0 size=1048576 2024-05-09T20:17:10.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.343+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~Z8RDL1NNGTPG.1 size=229252 2024-05-09T20:17:10.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.343+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=Z8RDL1NNGTPG tag=L2C3II3B8WS1 type=write size=2326404 2024-05-09T20:17:10.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.347+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=WC8N0QSVRLZP type=del 2024-05-09T20:17:10.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.351+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=P6JFBEBR68RR type=del size=408472 2024-05-09T20:17:10.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.351+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~PI9Q9MGWB86P.0 size=601608 2024-05-09T20:17:10.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.359+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=PI9Q9MGWB86P tag=D0K0WEKVEWV5 type=write size=1650184 2024-05-09T20:17:10.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.367+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~7MBGS980RROR.0 size=354906 2024-05-09T20:17:10.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.367+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=7MBGS980RROR tag=FSI7QJ6T12H4 type=write size=1403482 2024-05-09T20:17:10.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.371+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=O6JEM6VFK4UT type=del 2024-05-09T20:17:10.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.371+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=5VOT7OK94KA7 type=write size=67029 2024-05-09T20:17:10.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.375+0000 7fdc882e58c0 1 < canceled operation key=T5EL7X0P450L tag=MD57NH3ATOPJ type=write size=395695 2024-05-09T20:17:10.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.379+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=RVBX58IJ4U7J type=del size=3480422 2024-05-09T20:17:10.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.387+0000 7fdc882e58c0 1 < suggested update operation key=HPK2T0629B1P tag=OC5REMVUQTVV type=del size=1425518 2024-05-09T20:17:10.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.391+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=V6UAGJUMOD0B type=del size=2375572 2024-05-09T20:17:10.385 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.391+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~LYO1CVQJI78C.0 size=1048576 2024-05-09T20:17:10.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.395+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~LYO1CVQJI78C.1 size=1048576 2024-05-09T20:17:10.390 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.399+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~LYO1CVQJI78C.2 size=643132 2024-05-09T20:17:10.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.403+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=LYO1CVQJI78C tag=2VHWT0Q5L7JZ type=write size=3788860 2024-05-09T20:17:10.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.407+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=9V9VYB0BNRUT type=del size=3286448 2024-05-09T20:17:10.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.407+0000 7fdc882e58c0 1 < completed delete operation key=VZ7XOAT899GU tag=6GSKSHQUAJ1P type=del 2024-05-09T20:17:10.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.411+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=199PV2H04TG6 type=del size=1915522 2024-05-09T20:17:10.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.415+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~RQ9EXA199EI9.0 size=1048576 2024-05-09T20:17:10.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.415+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~RQ9EXA199EI9.1 size=1048576 2024-05-09T20:17:10.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.419+0000 7fdc882e58c0 1 < completed part key=_multipart_TQ5ZHATLNVPP.2~RQ9EXA199EI9.2 size=282717 2024-05-09T20:17:10.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.427+0000 7fdc882e58c0 1 < completed multipart upload key=TQ5ZHATLNVPP upload=RQ9EXA199EI9 tag=UT1NOAQKRTF0 type=write size=3428445 2024-05-09T20:17:10.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.431+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=IQYBQLAR9MEZ type=del size=1348654 2024-05-09T20:17:10.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.435+0000 7fdc882e58c0 1 < completed write operation key=JS90W6PIREMD tag=UG9FIYAU5DH4 type=write size=32350 2024-05-09T20:17:10.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.435+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=KVPL3UCSZM86 type=del size=4004385 2024-05-09T20:17:10.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.439+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=8B43BLVEC6ZW type=del 2024-05-09T20:17:10.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.443+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=I66MJ2Q1ZSAA type=del size=861847 2024-05-09T20:17:10.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.447+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=3EKYCK0Q4XJY type=del 2024-05-09T20:17:10.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.451+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=OEZXX8FC34M1 type=del size=3821053 2024-05-09T20:17:10.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.451+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=OPWZGNM52FJ6 type=del 2024-05-09T20:17:10.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.455+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=INIQKTEL82NQ type=del size=3957563 2024-05-09T20:17:10.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.459+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=KLDYANNEOIQW type=del 2024-05-09T20:17:10.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.463+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=INAHPPZTWNPN type=del size=1661569 2024-05-09T20:17:10.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.467+0000 7fdc882e58c0 1 < canceled operation key=SNPNT5JFJOUS tag=M1G5GOQHYGKT type=del size=2485783 2024-05-09T20:17:10.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.471+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~A1NNVKUPTE40.0 size=1048576 2024-05-09T20:17:10.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.475+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~A1NNVKUPTE40.1 size=1048576 2024-05-09T20:17:10.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.479+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~A1NNVKUPTE40.2 size=19726 2024-05-09T20:17:10.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.479+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=A1NNVKUPTE40 tag=Z4N9S1C8X0WG type=write size=3165454 2024-05-09T20:17:10.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.479+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~8K4TXA1J4QZT.0 size=1048576 2024-05-09T20:17:10.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.479+0000 7fdc882e58c0 1 < canceled part key=_multipart_54R9XPO62J0M.2~8K4TXA1J4QZT.1 size=999317 2024-05-09T20:17:10.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.487+0000 7fdc882e58c0 1 < canceled multipart upload key=54R9XPO62J0M upload=8K4TXA1J4QZT tag=G71SO5EFSNS1 type=write size=3096469 2024-05-09T20:17:10.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.491+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~YD4JJX16X06M.0 size=1048576 2024-05-09T20:17:10.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.495+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~YD4JJX16X06M.1 size=1048576 2024-05-09T20:17:10.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.499+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~YD4JJX16X06M.2 size=914944 2024-05-09T20:17:10.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.499+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=YD4JJX16X06M tag=R036YEC7QQUQ type=write size=4060672 2024-05-09T20:17:10.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.503+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~Z8RDL1NNGTPG.0 size=1048576 2024-05-09T20:17:10.497 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.503+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~Z8RDL1NNGTPG.1 size=229252 2024-05-09T20:17:10.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.511+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=Z8RDL1NNGTPG tag=L2C3II3B8WS1 type=write size=2326404 2024-05-09T20:17:10.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.515+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~XLBSHS6OW8QW.0 size=1048576 2024-05-09T20:17:10.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.519+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~XLBSHS6OW8QW.1 size=1048576 2024-05-09T20:17:10.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.523+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~XLBSHS6OW8QW.2 size=592433 2024-05-09T20:17:10.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.523+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=XLBSHS6OW8QW tag=ETPPGGIQSKJU type=write size=3738161 2024-05-09T20:17:10.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.527+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=P6JFBEBR68RR type=del 2024-05-09T20:17:10.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.531+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~MSFTNHSIMW80.0 size=1048576 2024-05-09T20:17:10.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.535+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~MSFTNHSIMW80.1 size=1048576 2024-05-09T20:17:10.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.539+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~MSFTNHSIMW80.2 size=835809 2024-05-09T20:17:10.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.539+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=MSFTNHSIMW80 tag=I1K1HO7Y30OK type=write size=3981537 2024-05-09T20:17:10.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.539+0000 7fdc882e58c0 1 < completed part key=_multipart_346HV1XQKCOZ.2~7MBGS980RROR.0 size=354906 2024-05-09T20:17:10.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.547+0000 7fdc882e58c0 1 < completed multipart upload key=346HV1XQKCOZ upload=7MBGS980RROR tag=FSI7QJ6T12H4 type=write size=1403482 2024-05-09T20:17:10.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.551+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~7S8MCHPR0HUL.0 size=639144 2024-05-09T20:17:10.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.551+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=7S8MCHPR0HUL tag=065IZ8U6S70X type=write size=1687720 2024-05-09T20:17:10.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.555+0000 7fdc882e58c0 1 < completed write operation key=BS0T1OQOBS67 tag=5VOT7OK94KA7 type=write size=67029 2024-05-09T20:17:10.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.559+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=SXZ5GA6YL28O type=del size=3129333 2024-05-09T20:17:10.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.563+0000 7fdc882e58c0 1 < canceled operation key=IZPPFLWVPRKO tag=RVBX58IJ4U7J type=del size=3480422 2024-05-09T20:17:10.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.567+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~VNM1GGGEKZNJ.0 size=1048576 2024-05-09T20:17:10.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.571+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~VNM1GGGEKZNJ.1 size=1048576 2024-05-09T20:17:10.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.575+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~VNM1GGGEKZNJ.2 size=87352 2024-05-09T20:17:10.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.575+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=VNM1GGGEKZNJ tag=07NWE33P25CE type=write size=3233080 2024-05-09T20:17:10.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.579+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=V6UAGJUMOD0B type=del 2024-05-09T20:17:10.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.579+0000 7fdc882e58c0 1 > prepared operation key=8FFZZDURUWM1 tag=ODDOWLZZEIDH type=del size=3229535 2024-05-09T20:17:10.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.583+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=9V9VYB0BNRUT type=del 2024-05-09T20:17:10.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.587+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=ITBT3T87CCIY type=del size=2789665 2024-05-09T20:17:10.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.591+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=199PV2H04TG6 type=del 2024-05-09T20:17:10.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.595+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~163VIXQE6CMU.0 size=1048576 2024-05-09T20:17:10.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.599+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~163VIXQE6CMU.1 size=1048576 2024-05-09T20:17:10.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.603+0000 7fdc882e58c0 1 > prepared part key=_multipart_54R9XPO62J0M.2~163VIXQE6CMU.2 size=620180 2024-05-09T20:17:10.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.603+0000 7fdc882e58c0 1 > prepared multipart upload key=54R9XPO62J0M upload=163VIXQE6CMU tag=7ZF9FMKYGM1L type=write size=3765908 2024-05-09T20:17:10.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.607+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=IQYBQLAR9MEZ type=del 2024-05-09T20:17:10.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.607+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=IB6ZT8AJ5HDE type=del size=3068885 2024-05-09T20:17:10.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.611+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=KVPL3UCSZM86 type=del 2024-05-09T20:17:10.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.619+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~W28CPABZURT3.0 size=66936 2024-05-09T20:17:10.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.619+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=W28CPABZURT3 tag=4991T3SUWWN4 type=write size=1115512 2024-05-09T20:17:10.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.627+0000 7fdc882e58c0 1 < suggested remove operation key=BS0T1OQOBS67 tag=I66MJ2Q1ZSAA type=del 2024-05-09T20:17:10.621 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.627+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=2M415SUYZ63Q type=write size=139683 2024-05-09T20:17:10.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.631+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=OEZXX8FC34M1 type=del 2024-05-09T20:17:10.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.639+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~5O5U6FM0XUSG.0 size=1048576 2024-05-09T20:17:10.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.639+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~5O5U6FM0XUSG.1 size=503493 2024-05-09T20:17:10.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.639+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=5O5U6FM0XUSG tag=4GV9RKKWCHAI type=write size=2600645 2024-05-09T20:17:10.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.643+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=INIQKTEL82NQ type=del 2024-05-09T20:17:10.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.647+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=7ST659H4I10E type=write size=168023 2024-05-09T20:17:10.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.651+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=INAHPPZTWNPN type=del 2024-05-09T20:17:10.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.655+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=3850PJ2CDAKA type=del size=3325576 2024-05-09T20:17:10.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.659+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~A1NNVKUPTE40.0 size=1048576 2024-05-09T20:17:10.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.663+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~A1NNVKUPTE40.1 size=1048576 2024-05-09T20:17:10.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.663+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~A1NNVKUPTE40.2 size=19726 2024-05-09T20:17:10.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.671+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=A1NNVKUPTE40 tag=Z4N9S1C8X0WG type=write size=3165454 2024-05-09T20:17:10.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.671+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=G3LJ9ZABSZX0 type=write size=656354 2024-05-09T20:17:10.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.675+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~YD4JJX16X06M.0 size=1048576 2024-05-09T20:17:10.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.679+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~YD4JJX16X06M.1 size=1048576 2024-05-09T20:17:10.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.683+0000 7fdc882e58c0 1 < completed part key=_multipart_NNGWUSCETMH4.2~YD4JJX16X06M.2 size=914944 2024-05-09T20:17:10.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.687+0000 7fdc882e58c0 1 < completed multipart upload key=NNGWUSCETMH4 upload=YD4JJX16X06M tag=R036YEC7QQUQ type=write size=4060672 2024-05-09T20:17:10.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.691+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=06QSYWH4QHSS type=del size=175493 2024-05-09T20:17:10.687 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.695+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~XLBSHS6OW8QW.0 size=1048576 2024-05-09T20:17:10.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.699+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~XLBSHS6OW8QW.1 size=1048576 2024-05-09T20:17:10.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.699+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~XLBSHS6OW8QW.2 size=592433 2024-05-09T20:17:10.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.707+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=XLBSHS6OW8QW tag=ETPPGGIQSKJU type=write size=3738161 2024-05-09T20:17:10.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.711+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~P1ZFBX5AO61X.0 size=1048576 2024-05-09T20:17:10.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.715+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~P1ZFBX5AO61X.1 size=1048576 2024-05-09T20:17:10.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.719+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~P1ZFBX5AO61X.2 size=592436 2024-05-09T20:17:10.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.719+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=P1ZFBX5AO61X tag=9BRQOWR6N1KU type=write size=3738164 2024-05-09T20:17:10.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.719+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~MSFTNHSIMW80.0 size=1048576 2024-05-09T20:17:10.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.723+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~MSFTNHSIMW80.1 size=1048576 2024-05-09T20:17:10.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.727+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~MSFTNHSIMW80.2 size=835809 2024-05-09T20:17:10.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.735+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=MSFTNHSIMW80 tag=I1K1HO7Y30OK type=write size=3981537 2024-05-09T20:17:10.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.739+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=CKHK7IVEKCHV type=del size=2378899 2024-05-09T20:17:10.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.743+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~7S8MCHPR0HUL.0 size=639144 2024-05-09T20:17:10.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.747+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=7S8MCHPR0HUL tag=065IZ8U6S70X type=write size=1687720 2024-05-09T20:17:10.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.751+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=D51TSL3UREY2 type=del size=1063186 2024-05-09T20:17:10.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.755+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=SXZ5GA6YL28O type=del 2024-05-09T20:17:10.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.755+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=N2PPIOCU2OIV type=write size=1011857 2024-05-09T20:17:10.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.759+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~VNM1GGGEKZNJ.0 size=1048576 2024-05-09T20:17:10.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.763+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~VNM1GGGEKZNJ.1 size=1048576 2024-05-09T20:17:10.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.767+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~VNM1GGGEKZNJ.2 size=87352 2024-05-09T20:17:10.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.771+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=VNM1GGGEKZNJ tag=07NWE33P25CE type=write size=3233080 2024-05-09T20:17:10.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.775+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~UTGUO3XVDZZQ.0 size=722151 2024-05-09T20:17:10.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.775+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=UTGUO3XVDZZQ tag=OIGH476RLS2W type=write size=1770727 2024-05-09T20:17:10.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.779+0000 7fdc882e58c0 1 < completed delete operation key=8FFZZDURUWM1 tag=ODDOWLZZEIDH type=del 2024-05-09T20:17:10.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.783+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=S3ZW5Y8R8870 type=del size=244709 2024-05-09T20:17:10.779 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.787+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=ITBT3T87CCIY type=del 2024-05-09T20:17:10.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.791+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=05Y1S5X7WL3U type=write size=120276 2024-05-09T20:17:10.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.791+0000 7fdc882e58c0 1 < canceled part key=_multipart_54R9XPO62J0M.2~163VIXQE6CMU.0 size=1048576 2024-05-09T20:17:10.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.795+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~163VIXQE6CMU.1 size=1048576 2024-05-09T20:17:10.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.795+0000 7fdc882e58c0 1 < completed part key=_multipart_54R9XPO62J0M.2~163VIXQE6CMU.2 size=620180 2024-05-09T20:17:10.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.803+0000 7fdc882e58c0 1 < canceled multipart upload key=54R9XPO62J0M upload=163VIXQE6CMU tag=7ZF9FMKYGM1L type=write size=3765908 2024-05-09T20:17:10.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.807+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=NOSOET5WIMRZ type=del size=3397827 2024-05-09T20:17:10.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.811+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=IB6ZT8AJ5HDE type=del 2024-05-09T20:17:10.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.815+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=G6F9BEETKACS type=del size=1921061 2024-05-09T20:17:10.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.819+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~W28CPABZURT3.0 size=66936 2024-05-09T20:17:10.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.827+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=W28CPABZURT3 tag=4991T3SUWWN4 type=write size=1115512 2024-05-09T20:17:10.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.831+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~OHQWF78LQKS7.0 size=1048576 2024-05-09T20:17:10.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.835+0000 7fdc882e58c0 1 > prepared part key=_multipart_NNGWUSCETMH4.2~OHQWF78LQKS7.1 size=333917 2024-05-09T20:17:10.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.835+0000 7fdc882e58c0 1 > prepared multipart upload key=NNGWUSCETMH4 upload=OHQWF78LQKS7 tag=4WV2VM3GAGCU type=write size=2431069 2024-05-09T20:17:10.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.839+0000 7fdc882e58c0 1 < completed write operation key=CNV7NHP27JZ7 tag=2M415SUYZ63Q type=write size=139683 2024-05-09T20:17:10.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.843+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~F2VUG1I16FKT.0 size=434190 2024-05-09T20:17:10.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.843+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=F2VUG1I16FKT tag=6YC66CJB02OT type=write size=1482766 2024-05-09T20:17:10.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.847+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~5O5U6FM0XUSG.0 size=1048576 2024-05-09T20:17:10.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.851+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~5O5U6FM0XUSG.1 size=503493 2024-05-09T20:17:10.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.855+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=5O5U6FM0XUSG tag=4GV9RKKWCHAI type=write size=2600645 2024-05-09T20:17:10.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.863+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~MX263CQ7E9PC.0 size=1048576 2024-05-09T20:17:10.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.863+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~MX263CQ7E9PC.1 size=1048576 2024-05-09T20:17:10.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.867+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~MX263CQ7E9PC.2 size=604018 2024-05-09T20:17:10.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.867+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=MX263CQ7E9PC tag=V08YNY9PD0XY type=write size=3749746 2024-05-09T20:17:10.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.875+0000 7fdc882e58c0 1 < suggested remove operation key=IZPPFLWVPRKO tag=7ST659H4I10E type=write 2024-05-09T20:17:10.872 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.879+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~DKI5M0L63MBP.0 size=1048576 2024-05-09T20:17:10.875 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.883+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~DKI5M0L63MBP.1 size=1048576 2024-05-09T20:17:10.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.883+0000 7fdc882e58c0 1 > prepared part key=_multipart_HPK2T0629B1P.2~DKI5M0L63MBP.2 size=43213 2024-05-09T20:17:10.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.883+0000 7fdc882e58c0 1 > prepared multipart upload key=HPK2T0629B1P upload=DKI5M0L63MBP tag=1UQ2F3Z6BKMO type=write size=3188941 2024-05-09T20:17:10.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.887+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=3850PJ2CDAKA type=del 2024-05-09T20:17:10.884 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.891+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=LP3SONVLWUVQ type=del size=2926928 2024-05-09T20:17:10.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.895+0000 7fdc882e58c0 1 < completed write operation key=P2DNA4DGI54P tag=G3LJ9ZABSZX0 type=write size=656354 2024-05-09T20:17:10.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.899+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~W5YMUBDM91CV.0 size=1048576 2024-05-09T20:17:10.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.903+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~W5YMUBDM91CV.1 size=135424 2024-05-09T20:17:10.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.903+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=W5YMUBDM91CV tag=VCGJ08P83TP2 type=write size=2232576 2024-05-09T20:17:10.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.907+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=06QSYWH4QHSS type=del 2024-05-09T20:17:10.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.911+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~XJO94GMXWLEE.0 size=1048576 2024-05-09T20:17:10.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.915+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~XJO94GMXWLEE.1 size=1048576 2024-05-09T20:17:10.911 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.919+0000 7fdc882e58c0 1 > prepared part key=_multipart_J2Q86HKRMAKE.2~XJO94GMXWLEE.2 size=69545 2024-05-09T20:17:10.911 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.919+0000 7fdc882e58c0 1 > prepared multipart upload key=J2Q86HKRMAKE upload=XJO94GMXWLEE tag=9VA4V0JYH1J5 type=write size=3215273 2024-05-09T20:17:10.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.923+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~P1ZFBX5AO61X.0 size=1048576 2024-05-09T20:17:10.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.923+0000 7fdc882e58c0 1 < canceled part key=_multipart_T5EL7X0P450L.2~P1ZFBX5AO61X.1 size=1048576 2024-05-09T20:17:10.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.923+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~P1ZFBX5AO61X.2 size=592436 2024-05-09T20:17:10.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.931+0000 7fdc882e58c0 1 < canceled multipart upload key=T5EL7X0P450L upload=P1ZFBX5AO61X tag=9BRQOWR6N1KU type=write size=3738164 2024-05-09T20:17:10.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.935+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=GPB45CRBM5VA type=del size=2106842 2024-05-09T20:17:10.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.939+0000 7fdc882e58c0 1 < completed delete operation key=AJANS5ADJO5B tag=CKHK7IVEKCHV type=del 2024-05-09T20:17:10.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.943+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=Q2TZ1S9TU3KF type=del size=1895616 2024-05-09T20:17:10.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.947+0000 7fdc882e58c0 1 < canceled operation key=WC5YH0OU51WX tag=D51TSL3UREY2 type=del size=1063186 2024-05-09T20:17:10.945 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.951+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~L60QKT4Z9DT1.0 size=1048576 2024-05-09T20:17:10.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.955+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~L60QKT4Z9DT1.1 size=423617 2024-05-09T20:17:10.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.955+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=L60QKT4Z9DT1 tag=KY55EIVTD3VD type=write size=2520769 2024-05-09T20:17:10.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.955+0000 7fdc882e58c0 -1 < no bi entry to suggest for operation key=IZPPFLWVPRKO tag=N2PPIOCU2OIV type=write size=1011857: (2) No such file or directory 2024-05-09T20:17:10.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.963+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~UAPBQX1AUEF2.0 size=1048576 2024-05-09T20:17:10.959 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.967+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~UAPBQX1AUEF2.1 size=1048576 2024-05-09T20:17:10.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.971+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~UAPBQX1AUEF2.2 size=640107 2024-05-09T20:17:10.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.971+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=UAPBQX1AUEF2 tag=DKCA4T4629IY type=write size=3785835 2024-05-09T20:17:10.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.975+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~UTGUO3XVDZZQ.0 size=722151 2024-05-09T20:17:10.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.979+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=UTGUO3XVDZZQ tag=OIGH476RLS2W type=write size=1770727 2024-05-09T20:17:10.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.987+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~J95DC6PZM05U.0 size=978198 2024-05-09T20:17:10.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.987+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=J95DC6PZM05U tag=PS2666PDMGQ9 type=write size=2026774 2024-05-09T20:17:10.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.991+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=S3ZW5Y8R8870 type=del 2024-05-09T20:17:10.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.995+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=E4PDT2KEA4SB type=del size=1496826 2024-05-09T20:17:10.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:10.999+0000 7fdc882e58c0 1 < suggested remove operation key=54R9XPO62J0M tag=05Y1S5X7WL3U type=write 2024-05-09T20:17:10.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.007+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~FI6NC9L3PXG0.0 size=924177 2024-05-09T20:17:10.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.007+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=FI6NC9L3PXG0 tag=OBB1CYJIJ9H2 type=write size=1972753 2024-05-09T20:17:11.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.011+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=NOSOET5WIMRZ type=del 2024-05-09T20:17:11.006 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.015+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=10F0EC2WRLAF type=write size=913051 2024-05-09T20:17:11.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.019+0000 7fdc882e58c0 1 < completed delete operation key=HPK2T0629B1P tag=G6F9BEETKACS type=del 2024-05-09T20:17:11.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.023+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~DVI8V03Z8LYU.0 size=1048576 2024-05-09T20:17:11.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.027+0000 7fdc882e58c0 1 > prepared part key=_multipart_AJANS5ADJO5B.2~DVI8V03Z8LYU.1 size=472580 2024-05-09T20:17:11.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.027+0000 7fdc882e58c0 1 > prepared multipart upload key=AJANS5ADJO5B upload=DVI8V03Z8LYU tag=SN2C1QAQVSA5 type=write size=2569732 2024-05-09T20:17:11.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.027+0000 7fdc882e58c0 1 < canceled part key=_multipart_NNGWUSCETMH4.2~OHQWF78LQKS7.0 size=1048576 2024-05-09T20:17:11.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.027+0000 7fdc882e58c0 1 < canceled part key=_multipart_NNGWUSCETMH4.2~OHQWF78LQKS7.1 size=333917 2024-05-09T20:17:11.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.031+0000 7fdc882e58c0 1 < canceled multipart upload key=NNGWUSCETMH4 upload=OHQWF78LQKS7 tag=4WV2VM3GAGCU type=write size=2431069 2024-05-09T20:17:11.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.035+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=CSHA0AP23NX8 type=del size=770889 2024-05-09T20:17:11.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.039+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~F2VUG1I16FKT.0 size=434190 2024-05-09T20:17:11.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.043+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=F2VUG1I16FKT tag=6YC66CJB02OT type=write size=1482766 2024-05-09T20:17:11.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.047+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=GTJ49BZ2QNXW type=del size=3650583 2024-05-09T20:17:11.044 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.051+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~MX263CQ7E9PC.0 size=1048576 2024-05-09T20:17:11.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.059+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~MX263CQ7E9PC.1 size=1048576 2024-05-09T20:17:11.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.059+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~MX263CQ7E9PC.2 size=604018 2024-05-09T20:17:11.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.067+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=MX263CQ7E9PC tag=V08YNY9PD0XY type=write size=3749746 2024-05-09T20:17:11.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.071+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~IQGXPBS48LGD.0 size=828639 2024-05-09T20:17:11.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.071+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=IQGXPBS48LGD tag=L06DCPW2CIE4 type=write size=1877215 2024-05-09T20:17:11.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.075+0000 7fdc882e58c0 1 < completed part key=_multipart_HPK2T0629B1P.2~DKI5M0L63MBP.0 size=1048576 2024-05-09T20:17:11.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.075+0000 7fdc882e58c0 1 < canceled part key=_multipart_HPK2T0629B1P.2~DKI5M0L63MBP.1 size=1048576 2024-05-09T20:17:11.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.075+0000 7fdc882e58c0 1 < canceled part key=_multipart_HPK2T0629B1P.2~DKI5M0L63MBP.2 size=43213 2024-05-09T20:17:11.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.083+0000 7fdc882e58c0 1 < canceled multipart upload key=HPK2T0629B1P upload=DKI5M0L63MBP tag=1UQ2F3Z6BKMO type=write size=3188941 2024-05-09T20:17:11.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.083+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=38H6LO8BGDQX type=del size=1347604 2024-05-09T20:17:11.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.087+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=LP3SONVLWUVQ type=del 2024-05-09T20:17:11.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.091+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=U44MY88F7W77 type=del size=2146167 2024-05-09T20:17:11.088 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.095+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~W5YMUBDM91CV.0 size=1048576 2024-05-09T20:17:11.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.099+0000 7fdc882e58c0 1 < completed part key=_multipart_CNV7NHP27JZ7.2~W5YMUBDM91CV.1 size=135424 2024-05-09T20:17:11.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.103+0000 7fdc882e58c0 1 < completed multipart upload key=CNV7NHP27JZ7 upload=W5YMUBDM91CV tag=VCGJ08P83TP2 type=write size=2232576 2024-05-09T20:17:11.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.107+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=XMU7FMQTU8TP type=del size=3007061 2024-05-09T20:17:11.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.107+0000 7fdc882e58c0 1 < canceled part key=_multipart_J2Q86HKRMAKE.2~XJO94GMXWLEE.0 size=1048576 2024-05-09T20:17:11.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.111+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~XJO94GMXWLEE.1 size=1048576 2024-05-09T20:17:11.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.115+0000 7fdc882e58c0 1 < completed part key=_multipart_J2Q86HKRMAKE.2~XJO94GMXWLEE.2 size=69545 2024-05-09T20:17:11.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.119+0000 7fdc882e58c0 1 < canceled multipart upload key=J2Q86HKRMAKE upload=XJO94GMXWLEE tag=9VA4V0JYH1J5 type=write size=3215273 2024-05-09T20:17:11.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.123+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=2XT14NB2L2CJ type=write size=962847 2024-05-09T20:17:11.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.127+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=GPB45CRBM5VA type=del 2024-05-09T20:17:11.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.135+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~4KMSLE0EWTHK.0 size=1048576 2024-05-09T20:17:11.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.135+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~4KMSLE0EWTHK.1 size=633547 2024-05-09T20:17:11.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.135+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=4KMSLE0EWTHK tag=21Z29NHX5RPE type=write size=2730699 2024-05-09T20:17:11.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.139+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=Q2TZ1S9TU3KF type=del 2024-05-09T20:17:11.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.147+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~I9TN3WZ3E296.0 size=1048576 2024-05-09T20:17:11.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.151+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~I9TN3WZ3E296.1 size=1048576 2024-05-09T20:17:11.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.155+0000 7fdc882e58c0 1 > prepared part key=_multipart_5NEOR8OOFAFZ.2~I9TN3WZ3E296.2 size=212912 2024-05-09T20:17:11.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.155+0000 7fdc882e58c0 1 > prepared multipart upload key=5NEOR8OOFAFZ upload=I9TN3WZ3E296 tag=XHCALL41CXK3 type=write size=3358640 2024-05-09T20:17:11.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.159+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~L60QKT4Z9DT1.0 size=1048576 2024-05-09T20:17:11.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.159+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~L60QKT4Z9DT1.1 size=423617 2024-05-09T20:17:11.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.167+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=L60QKT4Z9DT1 tag=KY55EIVTD3VD type=write size=2520769 2024-05-09T20:17:11.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.171+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=Z2OO592QRHEH type=del size=3309584 2024-05-09T20:17:11.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.175+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~UAPBQX1AUEF2.0 size=1048576 2024-05-09T20:17:11.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.179+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~UAPBQX1AUEF2.1 size=1048576 2024-05-09T20:17:11.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.179+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~UAPBQX1AUEF2.2 size=640107 2024-05-09T20:17:11.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.183+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=UAPBQX1AUEF2 tag=DKCA4T4629IY type=write size=3785835 2024-05-09T20:17:11.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.187+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=UB4IGRJZY6OG type=del size=906192 2024-05-09T20:17:11.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.191+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~J95DC6PZM05U.0 size=978198 2024-05-09T20:17:11.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.199+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=J95DC6PZM05U tag=PS2666PDMGQ9 type=write size=2026774 2024-05-09T20:17:11.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.203+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=JAWVJH3R19RI type=del size=554633 2024-05-09T20:17:11.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.207+0000 7fdc882e58c0 1 < suggested update operation key=G0DX1F257RLD tag=E4PDT2KEA4SB type=del size=1496826 2024-05-09T20:17:11.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.211+0000 7fdc882e58c0 1 > prepared operation key=EIMPFFK4I6NZ tag=YXDZLJ5IXI0H type=del size=236443 2024-05-09T20:17:11.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.215+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~FI6NC9L3PXG0.0 size=924177 2024-05-09T20:17:11.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.223+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=FI6NC9L3PXG0 tag=OBB1CYJIJ9H2 type=write size=1972753 2024-05-09T20:17:11.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.227+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=GOB7VYT0JSNZ type=del size=1766435 2024-05-09T20:17:11.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.231+0000 7fdc882e58c0 1 < completed write operation key=SNPNT5JFJOUS tag=10F0EC2WRLAF type=write size=913051 2024-05-09T20:17:11.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.235+0000 7fdc882e58c0 1 > prepared operation key=5NEOR8OOFAFZ tag=63T6SAD2DXVG type=del size=4120643 2024-05-09T20:17:11.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.239+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~DVI8V03Z8LYU.0 size=1048576 2024-05-09T20:17:11.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.243+0000 7fdc882e58c0 1 < completed part key=_multipart_AJANS5ADJO5B.2~DVI8V03Z8LYU.1 size=472580 2024-05-09T20:17:11.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.251+0000 7fdc882e58c0 1 < completed multipart upload key=AJANS5ADJO5B upload=DVI8V03Z8LYU tag=SN2C1QAQVSA5 type=write size=2569732 2024-05-09T20:17:11.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.255+0000 7fdc882e58c0 1 > prepared operation key=O07UHAIJZF0R tag=JVT0V1ZMFE7E type=del size=541513 2024-05-09T20:17:11.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.259+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=CSHA0AP23NX8 type=del 2024-05-09T20:17:11.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.263+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=OZIZSOTVWXJH type=del size=2842670 2024-05-09T20:17:11.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.267+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=GTJ49BZ2QNXW type=del 2024-05-09T20:17:11.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.271+0000 7fdc882e58c0 1 > prepared operation key=T5EL7X0P450L tag=8XEN561VLD3B type=del size=1264783 2024-05-09T20:17:11.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.275+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~IQGXPBS48LGD.0 size=828639 2024-05-09T20:17:11.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.279+0000 7fdc882e58c0 1 < completed multipart upload key=OGVA2778MJWU upload=IQGXPBS48LGD tag=L06DCPW2CIE4 type=write size=1877215 2024-05-09T20:17:11.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.283+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=OTU5CTETAY9F type=del size=3431631 2024-05-09T20:17:11.278 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.287+0000 7fdc882e58c0 1 < canceled operation key=IZPPFLWVPRKO tag=38H6LO8BGDQX type=del size=1347604 2024-05-09T20:17:11.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.287+0000 7fdc882e58c0 1 > prepared operation key=TWWOL5D8C1MQ tag=Q9YVRFNA7L3R type=del size=3888734 2024-05-09T20:17:11.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.295+0000 7fdc882e58c0 1 < suggested update operation key=54R9XPO62J0M tag=U44MY88F7W77 type=del size=2146167 2024-05-09T20:17:11.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.299+0000 7fdc882e58c0 1 > prepared operation key=54R9XPO62J0M tag=88V2BAMIC2VF type=write size=229761 2024-05-09T20:17:11.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.303+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=XMU7FMQTU8TP type=del 2024-05-09T20:17:11.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.307+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=Z3A3N5UT3URU type=del size=110675 2024-05-09T20:17:11.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.311+0000 7fdc882e58c0 1 < completed write operation key=AJANS5ADJO5B tag=2XT14NB2L2CJ type=write size=962847 2024-05-09T20:17:11.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.315+0000 7fdc882e58c0 1 > prepared operation key=OGVA2778MJWU tag=4ZVIRYXYSCI3 type=del size=4042382 2024-05-09T20:17:11.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.315+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~4KMSLE0EWTHK.0 size=1048576 2024-05-09T20:17:11.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.319+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~4KMSLE0EWTHK.1 size=633547 2024-05-09T20:17:11.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.327+0000 7fdc882e58c0 1 < completed multipart upload key=FNB9B18LIA8L upload=4KMSLE0EWTHK tag=21Z29NHX5RPE type=write size=2730699 2024-05-09T20:17:11.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.331+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=887TR5IFWJOR type=del size=2584647 2024-05-09T20:17:11.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.335+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~I9TN3WZ3E296.0 size=1048576 2024-05-09T20:17:11.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.335+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~I9TN3WZ3E296.1 size=1048576 2024-05-09T20:17:11.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.339+0000 7fdc882e58c0 1 < completed part key=_multipart_5NEOR8OOFAFZ.2~I9TN3WZ3E296.2 size=212912 2024-05-09T20:17:11.338 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.347+0000 7fdc882e58c0 1 < completed multipart upload key=5NEOR8OOFAFZ upload=I9TN3WZ3E296 tag=XHCALL41CXK3 type=write size=3358640 2024-05-09T20:17:11.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.351+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=8XP8LC37E6EC type=del size=3655841 2024-05-09T20:17:11.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.351+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=Z2OO592QRHEH type=del 2024-05-09T20:17:11.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.355+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=LIGJ7JJ8K2PQ type=del size=1850677 2024-05-09T20:17:11.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.359+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=UB4IGRJZY6OG type=del 2024-05-09T20:17:11.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.367+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~G3FR7DVYIBFP.0 size=1048576 2024-05-09T20:17:11.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.371+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~G3FR7DVYIBFP.1 size=1048576 2024-05-09T20:17:11.364 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.371+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~G3FR7DVYIBFP.2 size=648472 2024-05-09T20:17:11.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.371+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=G3FR7DVYIBFP tag=V0GLHOJS1OEQ type=write size=3794200 2024-05-09T20:17:11.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.375+0000 7fdc882e58c0 1 < canceled operation key=HPK2T0629B1P tag=JAWVJH3R19RI type=del size=554633 2024-05-09T20:17:11.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.383+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~YHI7M95SBFER.0 size=993518 2024-05-09T20:17:11.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.383+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=YHI7M95SBFER tag=LEWA3PT0MAD5 type=write size=2042094 2024-05-09T20:17:11.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.387+0000 7fdc882e58c0 1 < completed delete operation key=EIMPFFK4I6NZ tag=YXDZLJ5IXI0H type=del 2024-05-09T20:17:11.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.387+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=084MEY19LE4H type=del size=3718916 2024-05-09T20:17:11.385 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.391+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=GOB7VYT0JSNZ type=del 2024-05-09T20:17:11.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.403+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~1G13Y4UR5R1U.0 size=1048576 2024-05-09T20:17:11.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.407+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~1G13Y4UR5R1U.1 size=430386 2024-05-09T20:17:11.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.407+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=1G13Y4UR5R1U tag=8BKLW2EHJTTC type=write size=2527538 2024-05-09T20:17:11.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.411+0000 7fdc882e58c0 1 < completed delete operation key=5NEOR8OOFAFZ tag=63T6SAD2DXVG type=del 2024-05-09T20:17:11.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.415+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~W3U6IRDO1TP6.0 size=1048576 2024-05-09T20:17:11.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.419+0000 7fdc882e58c0 1 > prepared part key=_multipart_5H2GWM9LQELZ.2~W3U6IRDO1TP6.1 size=714992 2024-05-09T20:17:11.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.419+0000 7fdc882e58c0 1 > prepared multipart upload key=5H2GWM9LQELZ upload=W3U6IRDO1TP6 tag=CB526L25HWBW type=write size=2812144 2024-05-09T20:17:11.415 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.423+0000 7fdc882e58c0 1 < completed delete operation key=O07UHAIJZF0R tag=JVT0V1ZMFE7E type=del 2024-05-09T20:17:11.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.427+0000 7fdc882e58c0 1 > prepared operation key=NNGWUSCETMH4 tag=DI01RYH9DXAL type=del size=311651 2024-05-09T20:17:11.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.431+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=OZIZSOTVWXJH type=del 2024-05-09T20:17:11.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.435+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~86F207SZM3VA.0 size=32279 2024-05-09T20:17:11.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.435+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=86F207SZM3VA tag=3QIVYZYGK4D7 type=write size=1080855 2024-05-09T20:17:11.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.439+0000 7fdc882e58c0 1 < completed delete operation key=T5EL7X0P450L tag=8XEN561VLD3B type=del 2024-05-09T20:17:11.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.443+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=013CFYVWQ0DP type=del size=3673607 2024-05-09T20:17:11.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.447+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=OTU5CTETAY9F type=del 2024-05-09T20:17:11.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.447+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=SPKNL7CTIIRZ type=del size=2083485 2024-05-09T20:17:11.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.451+0000 7fdc882e58c0 1 < completed delete operation key=TWWOL5D8C1MQ tag=Q9YVRFNA7L3R type=del 2024-05-09T20:17:11.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.455+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=TGYET4BRXKMP type=del size=3338305 2024-05-09T20:17:11.450 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.459+0000 7fdc882e58c0 1 < completed write operation key=54R9XPO62J0M tag=88V2BAMIC2VF type=write size=229761 2024-05-09T20:17:11.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.463+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~O9HETYJQBWV5.0 size=1048576 2024-05-09T20:17:11.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.467+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~O9HETYJQBWV5.1 size=1048576 2024-05-09T20:17:11.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.467+0000 7fdc882e58c0 1 > prepared part key=_multipart_OGVA2778MJWU.2~O9HETYJQBWV5.2 size=81367 2024-05-09T20:17:11.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.467+0000 7fdc882e58c0 1 > prepared multipart upload key=OGVA2778MJWU upload=O9HETYJQBWV5 tag=CB8J8SINQJEP type=write size=3227095 2024-05-09T20:17:11.465 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.471+0000 7fdc882e58c0 1 < canceled operation key=5H2GWM9LQELZ tag=Z3A3N5UT3URU type=del size=110675 2024-05-09T20:17:11.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.475+0000 7fdc882e58c0 1 > prepared operation key=TQ5ZHATLNVPP tag=5PEHBLPPO1ZM type=del size=1716382 2024-05-09T20:17:11.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.483+0000 7fdc882e58c0 1 < suggested remove operation key=OGVA2778MJWU tag=4ZVIRYXYSCI3 type=del 2024-05-09T20:17:11.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.487+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=JUZCPK02L5FN type=del size=2362005 2024-05-09T20:17:11.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.487+0000 7fdc882e58c0 1 < completed delete operation key=G0DX1F257RLD tag=887TR5IFWJOR type=del 2024-05-09T20:17:11.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.491+0000 7fdc882e58c0 1 > prepared operation key=SNPNT5JFJOUS tag=GINHZRUDZK6P type=del size=487808 2024-05-09T20:17:11.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.495+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=8XP8LC37E6EC type=del 2024-05-09T20:17:11.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.499+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=FN5J84IOW2TY type=del size=437802 2024-05-09T20:17:11.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.503+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=LIGJ7JJ8K2PQ type=del 2024-05-09T20:17:11.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.507+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~G8X8ADGNIL1L.0 size=1048576 2024-05-09T20:17:11.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.511+0000 7fdc882e58c0 1 > prepared part key=_multipart_SNPNT5JFJOUS.2~G8X8ADGNIL1L.1 size=928287 2024-05-09T20:17:11.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.511+0000 7fdc882e58c0 1 > prepared multipart upload key=SNPNT5JFJOUS upload=G8X8ADGNIL1L tag=5ZNMHSYRARSQ type=write size=3025439 2024-05-09T20:17:11.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.515+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~G3FR7DVYIBFP.0 size=1048576 2024-05-09T20:17:11.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.515+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~G3FR7DVYIBFP.1 size=1048576 2024-05-09T20:17:11.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.519+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~G3FR7DVYIBFP.2 size=648472 2024-05-09T20:17:11.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.527+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=G3FR7DVYIBFP tag=V0GLHOJS1OEQ type=write size=3794200 2024-05-09T20:17:11.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.527+0000 7fdc882e58c0 1 > prepared operation key=SPXHCWZICCWY tag=UDYQAMXFZIKA type=del size=2048000 2024-05-09T20:17:11.525 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.531+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~YHI7M95SBFER.0 size=993518 2024-05-09T20:17:11.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.539+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=YHI7M95SBFER tag=LEWA3PT0MAD5 type=write size=2042094 2024-05-09T20:17:11.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.539+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=YGFIU7KHN4WM type=del size=372263 2024-05-09T20:17:11.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.543+0000 7fdc882e58c0 1 < completed delete operation key=BS0T1OQOBS67 tag=084MEY19LE4H type=del 2024-05-09T20:17:11.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.547+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~H9C1JG258FLY.0 size=1048576 2024-05-09T20:17:11.544 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.551+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~H9C1JG258FLY.1 size=1048576 2024-05-09T20:17:11.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.555+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~H9C1JG258FLY.2 size=373766 2024-05-09T20:17:11.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.555+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=H9C1JG258FLY tag=4MSHKEEJD8VX type=write size=3519494 2024-05-09T20:17:11.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.559+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~1G13Y4UR5R1U.0 size=1048576 2024-05-09T20:17:11.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.559+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~1G13Y4UR5R1U.1 size=430386 2024-05-09T20:17:11.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.563+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=1G13Y4UR5R1U tag=8BKLW2EHJTTC type=write size=2527538 2024-05-09T20:17:11.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.567+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=OXGZLO1RGC7K type=del size=491588 2024-05-09T20:17:11.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.571+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~W3U6IRDO1TP6.0 size=1048576 2024-05-09T20:17:11.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.575+0000 7fdc882e58c0 1 < completed part key=_multipart_5H2GWM9LQELZ.2~W3U6IRDO1TP6.1 size=714992 2024-05-09T20:17:11.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.579+0000 7fdc882e58c0 1 < completed multipart upload key=5H2GWM9LQELZ upload=W3U6IRDO1TP6 tag=CB526L25HWBW type=write size=2812144 2024-05-09T20:17:11.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.583+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~RR34HASI7BN1.0 size=65998 2024-05-09T20:17:11.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.583+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=RR34HASI7BN1 tag=I04604R4Q3SW type=write size=1114574 2024-05-09T20:17:11.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.587+0000 7fdc882e58c0 1 < completed delete operation key=NNGWUSCETMH4 tag=DI01RYH9DXAL type=del 2024-05-09T20:17:11.583 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.591+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=WFP3BVAL7IL4 type=del size=1621123 2024-05-09T20:17:11.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.595+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~86F207SZM3VA.0 size=32279 2024-05-09T20:17:11.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.599+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=86F207SZM3VA tag=3QIVYZYGK4D7 type=write size=1080855 2024-05-09T20:17:11.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.603+0000 7fdc882e58c0 1 > prepared operation key=EK78CNKUHNKO tag=LTONI585X335 type=del size=887250 2024-05-09T20:17:11.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.607+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=013CFYVWQ0DP type=del 2024-05-09T20:17:11.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.611+0000 7fdc882e58c0 1 > prepared operation key=AJANS5ADJO5B tag=QSRX7X3ZL1Q7 type=del size=1677276 2024-05-09T20:17:11.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.611+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=SPKNL7CTIIRZ type=del 2024-05-09T20:17:11.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.619+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~ILH25O8AJ8A0.0 size=1048576 2024-05-09T20:17:11.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.619+0000 7fdc882e58c0 1 > prepared part key=_multipart_T6T3IOW9VPA0.2~ILH25O8AJ8A0.1 size=643543 2024-05-09T20:17:11.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.619+0000 7fdc882e58c0 1 > prepared multipart upload key=T6T3IOW9VPA0 upload=ILH25O8AJ8A0 tag=YF6WJ36BHW6S type=write size=2740695 2024-05-09T20:17:11.616 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.623+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=TGYET4BRXKMP type=del 2024-05-09T20:17:11.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.627+0000 7fdc882e58c0 1 > prepared operation key=HPK2T0629B1P tag=TWUVOUTI0C8R type=del size=3839244 2024-05-09T20:17:11.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.631+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~O9HETYJQBWV5.0 size=1048576 2024-05-09T20:17:11.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.631+0000 7fdc882e58c0 1 < canceled part key=_multipart_OGVA2778MJWU.2~O9HETYJQBWV5.1 size=1048576 2024-05-09T20:17:11.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.631+0000 7fdc882e58c0 1 < completed part key=_multipart_OGVA2778MJWU.2~O9HETYJQBWV5.2 size=81367 2024-05-09T20:17:11.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.639+0000 7fdc882e58c0 1 < canceled multipart upload key=OGVA2778MJWU upload=O9HETYJQBWV5 tag=CB8J8SINQJEP type=write size=3227095 2024-05-09T20:17:11.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.639+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=6B1BML07I3XG type=write size=869651 2024-05-09T20:17:11.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.643+0000 7fdc882e58c0 1 < completed delete operation key=TQ5ZHATLNVPP tag=5PEHBLPPO1ZM type=del 2024-05-09T20:17:11.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.647+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=VP68583WWQYN type=del size=670783 2024-05-09T20:17:11.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.651+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=JUZCPK02L5FN type=del 2024-05-09T20:17:11.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.655+0000 7fdc882e58c0 1 > prepared operation key=CNV7NHP27JZ7 tag=T88OEYQ5FV24 type=del size=51203 2024-05-09T20:17:11.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.655+0000 7fdc882e58c0 1 < completed delete operation key=SNPNT5JFJOUS tag=GINHZRUDZK6P type=del 2024-05-09T20:17:11.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.659+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=FD8XTA70RNUZ type=write size=446601 2024-05-09T20:17:11.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.663+0000 7fdc882e58c0 1 < canceled operation key=HPK2T0629B1P tag=FN5J84IOW2TY type=del size=437802 2024-05-09T20:17:11.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.671+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~D4YXCT91W6EE.0 size=499786 2024-05-09T20:17:11.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.671+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=D4YXCT91W6EE tag=BYFLYFXNDYCQ type=write size=1548362 2024-05-09T20:17:11.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.671+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~G8X8ADGNIL1L.0 size=1048576 2024-05-09T20:17:11.668 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.675+0000 7fdc882e58c0 1 < completed part key=_multipart_SNPNT5JFJOUS.2~G8X8ADGNIL1L.1 size=928287 2024-05-09T20:17:11.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.679+0000 7fdc882e58c0 1 < completed multipart upload key=SNPNT5JFJOUS upload=G8X8ADGNIL1L tag=5ZNMHSYRARSQ type=write size=3025439 2024-05-09T20:17:11.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.687+0000 7fdc882e58c0 1 > prepared part key=_multipart_CNV7NHP27JZ7.2~03687F8UN4TJ.0 size=939361 2024-05-09T20:17:11.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.687+0000 7fdc882e58c0 1 > prepared multipart upload key=CNV7NHP27JZ7 upload=03687F8UN4TJ tag=ZMSG4H4M11D5 type=write size=1987937 2024-05-09T20:17:11.682 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.691+0000 7fdc882e58c0 1 < completed delete operation key=SPXHCWZICCWY tag=UDYQAMXFZIKA type=del 2024-05-09T20:17:11.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.695+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=NUFJ53O4WNSZ type=write size=112198 2024-05-09T20:17:11.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.695+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=YGFIU7KHN4WM type=del 2024-05-09T20:17:11.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.707+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~5TG1NAW58YRE.0 size=1048576 2024-05-09T20:17:11.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.711+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~5TG1NAW58YRE.1 size=1048576 2024-05-09T20:17:11.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.715+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~5TG1NAW58YRE.2 size=7256 2024-05-09T20:17:11.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.715+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=5TG1NAW58YRE tag=2I6529NRMI9U type=write size=3152984 2024-05-09T20:17:11.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.715+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~H9C1JG258FLY.0 size=1048576 2024-05-09T20:17:11.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.719+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~H9C1JG258FLY.1 size=1048576 2024-05-09T20:17:11.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.719+0000 7fdc882e58c0 1 < canceled part key=_multipart_SPXHCWZICCWY.2~H9C1JG258FLY.2 size=373766 2024-05-09T20:17:11.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.727+0000 7fdc882e58c0 1 < canceled multipart upload key=SPXHCWZICCWY upload=H9C1JG258FLY tag=4MSHKEEJD8VX type=write size=3519494 2024-05-09T20:17:11.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.731+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~KHKGMRGK31ZS.0 size=1048576 2024-05-09T20:17:11.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.735+0000 7fdc882e58c0 1 > prepared part key=_multipart_TWWOL5D8C1MQ.2~KHKGMRGK31ZS.1 size=196890 2024-05-09T20:17:11.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.735+0000 7fdc882e58c0 1 > prepared multipart upload key=TWWOL5D8C1MQ upload=KHKGMRGK31ZS tag=A5TU2ZTTRO5G type=write size=2294042 2024-05-09T20:17:11.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.739+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=OXGZLO1RGC7K type=del 2024-05-09T20:17:11.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.743+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~KZHXNCZ1Y96B.0 size=467869 2024-05-09T20:17:11.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.743+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=KZHXNCZ1Y96B tag=KPN2KJ08CEHM type=write size=1516445 2024-05-09T20:17:11.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.747+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~RR34HASI7BN1.0 size=65998 2024-05-09T20:17:11.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.755+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=RR34HASI7BN1 tag=I04604R4Q3SW type=write size=1114574 2024-05-09T20:17:11.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.759+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~6AH7E4KEGSZ3.0 size=1048576 2024-05-09T20:17:11.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.763+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~6AH7E4KEGSZ3.1 size=1048576 2024-05-09T20:17:11.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.767+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~6AH7E4KEGSZ3.2 size=160472 2024-05-09T20:17:11.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.767+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=6AH7E4KEGSZ3 tag=4XSH2HWK6DO8 type=write size=3306200 2024-05-09T20:17:11.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.767+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=WFP3BVAL7IL4 type=del 2024-05-09T20:17:11.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.771+0000 7fdc882e58c0 1 > prepared operation key=JS90W6PIREMD tag=9MNZ77ZGDAMO type=del size=308088 2024-05-09T20:17:11.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.779+0000 7fdc882e58c0 1 < suggested update operation key=EK78CNKUHNKO tag=LTONI585X335 type=del size=887250 2024-05-09T20:17:11.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.783+0000 7fdc882e58c0 1 > prepared operation key=QM53IENH9CX4 tag=EVQ28V7EEJF7 type=del size=1993417 2024-05-09T20:17:11.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.787+0000 7fdc882e58c0 1 < suggested remove operation key=AJANS5ADJO5B tag=QSRX7X3ZL1Q7 type=del 2024-05-09T20:17:11.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.791+0000 7fdc882e58c0 1 > prepared operation key=P2DNA4DGI54P tag=QA6MREYW4KW4 type=del size=3787673 2024-05-09T20:17:11.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.795+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~ILH25O8AJ8A0.0 size=1048576 2024-05-09T20:17:11.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.799+0000 7fdc882e58c0 1 < completed part key=_multipart_T6T3IOW9VPA0.2~ILH25O8AJ8A0.1 size=643543 2024-05-09T20:17:11.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.803+0000 7fdc882e58c0 1 < completed multipart upload key=T6T3IOW9VPA0 upload=ILH25O8AJ8A0 tag=YF6WJ36BHW6S type=write size=2740695 2024-05-09T20:17:11.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.807+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=U938719HCHJ0 type=del size=3250615 2024-05-09T20:17:11.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.811+0000 7fdc882e58c0 1 < suggested update operation key=HPK2T0629B1P tag=TWUVOUTI0C8R type=del size=3839244 2024-05-09T20:17:11.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.815+0000 7fdc882e58c0 1 > prepared operation key=BS0T1OQOBS67 tag=F020LD5P2C3S type=del size=3367277 2024-05-09T20:17:11.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.819+0000 7fdc882e58c0 1 < completed write operation key=5UMJFJRKUPKR tag=6B1BML07I3XG type=write size=869651 2024-05-09T20:17:11.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.823+0000 7fdc882e58c0 1 > prepared part key=_multipart_G0DX1F257RLD.2~W6T5CYZK2MGW.0 size=657345 2024-05-09T20:17:11.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.823+0000 7fdc882e58c0 1 > prepared multipart upload key=G0DX1F257RLD upload=W6T5CYZK2MGW tag=SCGBMEBN00SY type=write size=1705921 2024-05-09T20:17:11.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.827+0000 7fdc882e58c0 1 < completed delete operation key=5UMJFJRKUPKR tag=VP68583WWQYN type=del 2024-05-09T20:17:11.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.831+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=BV506NZ19MBZ type=del size=3211744 2024-05-09T20:17:11.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.835+0000 7fdc882e58c0 1 < completed delete operation key=CNV7NHP27JZ7 tag=T88OEYQ5FV24 type=del 2024-05-09T20:17:11.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.835+0000 7fdc882e58c0 1 > prepared operation key=IZPPFLWVPRKO tag=PO6TFR2ZGZ4B type=del size=3812496 2024-05-09T20:17:11.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.843+0000 7fdc882e58c0 1 < suggested remove operation key=5UMJFJRKUPKR tag=FD8XTA70RNUZ type=write 2024-05-09T20:17:11.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.847+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~7A479OKVI15L.0 size=1048576 2024-05-09T20:17:11.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.851+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~7A479OKVI15L.1 size=1048576 2024-05-09T20:17:11.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.851+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~7A479OKVI15L.2 size=928343 2024-05-09T20:17:11.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.851+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=7A479OKVI15L tag=VX9KKLPOY58T type=write size=4074071 2024-05-09T20:17:11.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.855+0000 7fdc882e58c0 1 < completed part key=_multipart_TWWOL5D8C1MQ.2~D4YXCT91W6EE.0 size=499786 2024-05-09T20:17:11.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.859+0000 7fdc882e58c0 1 < completed multipart upload key=TWWOL5D8C1MQ upload=D4YXCT91W6EE tag=BYFLYFXNDYCQ type=write size=1548362 2024-05-09T20:17:11.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.867+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~FAE0CXF35YH9.0 size=1048576 2024-05-09T20:17:11.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.867+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~FAE0CXF35YH9.1 size=1048576 2024-05-09T20:17:11.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.871+0000 7fdc882e58c0 1 > prepared part key=_multipart_FNB9B18LIA8L.2~FAE0CXF35YH9.2 size=656724 2024-05-09T20:17:11.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.871+0000 7fdc882e58c0 1 > prepared multipart upload key=FNB9B18LIA8L upload=FAE0CXF35YH9 tag=89SUNWU0W8K0 type=write size=3802452 2024-05-09T20:17:11.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.871+0000 7fdc882e58c0 1 < canceled part key=_multipart_CNV7NHP27JZ7.2~03687F8UN4TJ.0 size=939361 2024-05-09T20:17:11.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.875+0000 7fdc882e58c0 1 < canceled multipart upload key=CNV7NHP27JZ7 upload=03687F8UN4TJ tag=ZMSG4H4M11D5 type=write size=1987937 2024-05-09T20:17:11.870 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.879+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=JWZH259WA3R6 type=del size=1963661 2024-05-09T20:17:11.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.883+0000 7fdc882e58c0 1 < suggested remove operation key=J2Q86HKRMAKE tag=NUFJ53O4WNSZ type=write 2024-05-09T20:17:11.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.887+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=BNBTOJAFUWKC type=del size=3591215 2024-05-09T20:17:11.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.887+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~5TG1NAW58YRE.0 size=1048576 2024-05-09T20:17:11.884 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.891+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~5TG1NAW58YRE.1 size=1048576 2024-05-09T20:17:11.886 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.895+0000 7fdc882e58c0 1 < completed part key=_multipart_QM53IENH9CX4.2~5TG1NAW58YRE.2 size=7256 2024-05-09T20:17:11.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.899+0000 7fdc882e58c0 1 < completed multipart upload key=QM53IENH9CX4 upload=5TG1NAW58YRE tag=2I6529NRMI9U type=write size=3152984 2024-05-09T20:17:11.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.899+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=4KGZKPJ2JEOY type=del size=918172 2024-05-09T20:17:11.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.899+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~KHKGMRGK31ZS.0 size=1048576 2024-05-09T20:17:11.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.899+0000 7fdc882e58c0 1 < canceled part key=_multipart_TWWOL5D8C1MQ.2~KHKGMRGK31ZS.1 size=196890 2024-05-09T20:17:11.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.907+0000 7fdc882e58c0 1 < canceled multipart upload key=TWWOL5D8C1MQ upload=KHKGMRGK31ZS tag=A5TU2ZTTRO5G type=write size=2294042 2024-05-09T20:17:11.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.911+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~U8HAL85CPO4D.0 size=1048576 2024-05-09T20:17:11.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.911+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~U8HAL85CPO4D.1 size=1048576 2024-05-09T20:17:11.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.915+0000 7fdc882e58c0 1 > prepared part key=_multipart_SPXHCWZICCWY.2~U8HAL85CPO4D.2 size=782287 2024-05-09T20:17:11.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.915+0000 7fdc882e58c0 1 > prepared multipart upload key=SPXHCWZICCWY upload=U8HAL85CPO4D tag=VMO4MEG3O48L type=write size=3928015 2024-05-09T20:17:11.910 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.919+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~KZHXNCZ1Y96B.0 size=467869 2024-05-09T20:17:11.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.923+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=KZHXNCZ1Y96B tag=KPN2KJ08CEHM type=write size=1516445 2024-05-09T20:17:11.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.927+0000 7fdc882e58c0 1 > prepared operation key=WC5YH0OU51WX tag=GHUUX340S9XC type=del size=3420433 2024-05-09T20:17:11.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.927+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~6AH7E4KEGSZ3.0 size=1048576 2024-05-09T20:17:11.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.931+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~6AH7E4KEGSZ3.1 size=1048576 2024-05-09T20:17:11.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.931+0000 7fdc882e58c0 1 < completed part key=_multipart_WC5YH0OU51WX.2~6AH7E4KEGSZ3.2 size=160472 2024-05-09T20:17:11.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.939+0000 7fdc882e58c0 1 < completed multipart upload key=WC5YH0OU51WX upload=6AH7E4KEGSZ3 tag=4XSH2HWK6DO8 type=write size=3306200 2024-05-09T20:17:11.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.943+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~HZDXTHH2162N.0 size=1048576 2024-05-09T20:17:11.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.947+0000 7fdc882e58c0 1 > prepared part key=_multipart_VZ7XOAT899GU.2~HZDXTHH2162N.1 size=590736 2024-05-09T20:17:11.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.947+0000 7fdc882e58c0 1 > prepared multipart upload key=VZ7XOAT899GU upload=HZDXTHH2162N tag=J6LAI5DFCKHR type=write size=2687888 2024-05-09T20:17:11.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.947+0000 7fdc882e58c0 1 < completed delete operation key=JS90W6PIREMD tag=9MNZ77ZGDAMO type=del 2024-05-09T20:17:11.946 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.951+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~5ZPUWQIC5X4F.0 size=1048576 2024-05-09T20:17:11.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.955+0000 7fdc882e58c0 1 > prepared part key=_multipart_P2DNA4DGI54P.2~5ZPUWQIC5X4F.1 size=746982 2024-05-09T20:17:11.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.955+0000 7fdc882e58c0 1 > prepared multipart upload key=P2DNA4DGI54P upload=5ZPUWQIC5X4F tag=U1WN5QK3JMTW type=write size=2844134 2024-05-09T20:17:11.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.959+0000 7fdc882e58c0 1 < completed delete operation key=QM53IENH9CX4 tag=EVQ28V7EEJF7 type=del 2024-05-09T20:17:11.957 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.963+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~N49W41ZILR3I.0 size=21246 2024-05-09T20:17:11.957 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.963+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=N49W41ZILR3I tag=JBL3EA5MY1VK type=write size=1069822 2024-05-09T20:17:11.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.967+0000 7fdc882e58c0 1 < completed delete operation key=P2DNA4DGI54P tag=QA6MREYW4KW4 type=del 2024-05-09T20:17:11.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.971+0000 7fdc882e58c0 1 > prepared operation key=NYHR6GTPQYZS tag=QH5KLXQLTDA8 type=del size=202603 2024-05-09T20:17:11.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.975+0000 7fdc882e58c0 1 < suggested remove operation key=BS0T1OQOBS67 tag=U938719HCHJ0 type=del 2024-05-09T20:17:11.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.979+0000 7fdc882e58c0 1 > prepared operation key=T6T3IOW9VPA0 tag=DM272BWYYNEN type=del size=3192827 2024-05-09T20:17:11.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.983+0000 7fdc882e58c0 -1 < failed to complete operation key=BS0T1OQOBS67 tag=F020LD5P2C3S type=del size=3367277: (22) Invalid argument 2024-05-09T20:17:11.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.983+0000 7fdc882e58c0 1 > prepared operation key=5UMJFJRKUPKR tag=7VN42U9BEUE3 type=del size=2149714 2024-05-09T20:17:11.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.991+0000 7fdc882e58c0 1 < completed part key=_multipart_G0DX1F257RLD.2~W6T5CYZK2MGW.0 size=657345 2024-05-09T20:17:11.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.999+0000 7fdc882e58c0 1 < completed multipart upload key=G0DX1F257RLD upload=W6T5CYZK2MGW tag=SCGBMEBN00SY type=write size=1705921 2024-05-09T20:17:11.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:11.999+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=3F3QUZR40YMV type=write size=530967 2024-05-09T20:17:11.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.003+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=BV506NZ19MBZ type=del 2024-05-09T20:17:12.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.007+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~ZP3HCH3882FQ.0 size=1048576 2024-05-09T20:17:12.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.011+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~ZP3HCH3882FQ.1 size=1006826 2024-05-09T20:17:12.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.011+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=ZP3HCH3882FQ tag=O2ZJYGZ0O1VZ type=write size=3103978 2024-05-09T20:17:12.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.015+0000 7fdc882e58c0 1 < completed delete operation key=IZPPFLWVPRKO tag=PO6TFR2ZGZ4B type=del 2024-05-09T20:17:12.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.015+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=VSVVEJ7W9YBL type=del size=3055842 2024-05-09T20:17:12.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.019+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~7A479OKVI15L.0 size=1048576 2024-05-09T20:17:12.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.023+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~7A479OKVI15L.1 size=1048576 2024-05-09T20:17:12.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.027+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~7A479OKVI15L.2 size=928343 2024-05-09T20:17:12.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.031+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=7A479OKVI15L tag=VX9KKLPOY58T type=write size=4074071 2024-05-09T20:17:12.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.035+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~W3S2X0AUMDGA.0 size=1048576 2024-05-09T20:17:12.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.039+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~W3S2X0AUMDGA.1 size=1048576 2024-05-09T20:17:12.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.043+0000 7fdc882e58c0 1 > prepared part key=_multipart_5UMJFJRKUPKR.2~W3S2X0AUMDGA.2 size=659081 2024-05-09T20:17:12.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.043+0000 7fdc882e58c0 1 > prepared multipart upload key=5UMJFJRKUPKR upload=W3S2X0AUMDGA tag=QI13XSP7V064 type=write size=3804809 2024-05-09T20:17:12.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.047+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~FAE0CXF35YH9.0 size=1048576 2024-05-09T20:17:12.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.047+0000 7fdc882e58c0 1 < canceled part key=_multipart_FNB9B18LIA8L.2~FAE0CXF35YH9.1 size=1048576 2024-05-09T20:17:12.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.047+0000 7fdc882e58c0 1 < completed part key=_multipart_FNB9B18LIA8L.2~FAE0CXF35YH9.2 size=656724 2024-05-09T20:17:12.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.055+0000 7fdc882e58c0 1 < canceled multipart upload key=FNB9B18LIA8L upload=FAE0CXF35YH9 tag=89SUNWU0W8K0 type=write size=3802452 2024-05-09T20:17:12.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.059+0000 7fdc882e58c0 1 > prepared part key=_multipart_WC5YH0OU51WX.2~ZE21GD9ROLCZ.0 size=27784 2024-05-09T20:17:12.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.059+0000 7fdc882e58c0 1 > prepared multipart upload key=WC5YH0OU51WX upload=ZE21GD9ROLCZ tag=X0IETBT8XS1N type=write size=1076360 2024-05-09T20:17:12.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.063+0000 7fdc882e58c0 1 < completed delete operation key=5H2GWM9LQELZ tag=JWZH259WA3R6 type=del 2024-05-09T20:17:12.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.071+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~QJV3DXWPU1Z6.0 size=921688 2024-05-09T20:17:12.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.071+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=QJV3DXWPU1Z6 tag=KT5UL330IGX1 type=write size=1970264 2024-05-09T20:17:12.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.075+0000 7fdc882e58c0 1 < completed delete operation key=NBAECV9L85UK tag=BNBTOJAFUWKC type=del 2024-05-09T20:17:12.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.079+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~SP4HRWQKSYVW.0 size=1048576 2024-05-09T20:17:12.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.083+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~SP4HRWQKSYVW.1 size=1048576 2024-05-09T20:17:12.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.083+0000 7fdc882e58c0 1 > prepared part key=_multipart_QM53IENH9CX4.2~SP4HRWQKSYVW.2 size=89648 2024-05-09T20:17:12.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.083+0000 7fdc882e58c0 1 > prepared multipart upload key=QM53IENH9CX4 upload=SP4HRWQKSYVW tag=M9GWPP9ZB8PP type=write size=3235376 2024-05-09T20:17:12.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.091+0000 7fdc882e58c0 1 < suggested update operation key=T6T3IOW9VPA0 tag=4KGZKPJ2JEOY type=del size=918172 2024-05-09T20:17:12.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.099+0000 7fdc882e58c0 1 > prepared part key=_multipart_TQ5ZHATLNVPP.2~0ZYGXWIYZQ60.0 size=172354 2024-05-09T20:17:12.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.099+0000 7fdc882e58c0 1 > prepared multipart upload key=TQ5ZHATLNVPP upload=0ZYGXWIYZQ60 tag=20VSIM3OUDPT type=write size=1220930 2024-05-09T20:17:12.095 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.103+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~U8HAL85CPO4D.0 size=1048576 2024-05-09T20:17:12.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.107+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~U8HAL85CPO4D.1 size=1048576 2024-05-09T20:17:12.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.107+0000 7fdc882e58c0 1 < completed part key=_multipart_SPXHCWZICCWY.2~U8HAL85CPO4D.2 size=782287 2024-05-09T20:17:12.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.115+0000 7fdc882e58c0 1 < completed multipart upload key=SPXHCWZICCWY upload=U8HAL85CPO4D tag=VMO4MEG3O48L type=write size=3928015 2024-05-09T20:17:12.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.119+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=34RBR970R051 type=del size=583809 2024-05-09T20:17:12.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.123+0000 7fdc882e58c0 1 < completed delete operation key=WC5YH0OU51WX tag=GHUUX340S9XC type=del 2024-05-09T20:17:12.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.127+0000 7fdc882e58c0 1 > prepared operation key=VZ7XOAT899GU tag=R1LUDFW0JZVY type=del size=509379 2024-05-09T20:17:12.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.131+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~HZDXTHH2162N.0 size=1048576 2024-05-09T20:17:12.124 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.131+0000 7fdc882e58c0 1 < completed part key=_multipart_VZ7XOAT899GU.2~HZDXTHH2162N.1 size=590736 2024-05-09T20:17:12.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.139+0000 7fdc882e58c0 1 < completed multipart upload key=VZ7XOAT899GU upload=HZDXTHH2162N tag=J6LAI5DFCKHR type=write size=2687888 2024-05-09T20:17:12.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.143+0000 7fdc882e58c0 1 > prepared part key=_multipart_O07UHAIJZF0R.2~M9FCIGYESWCZ.0 size=1041139 2024-05-09T20:17:12.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.143+0000 7fdc882e58c0 1 > prepared multipart upload key=O07UHAIJZF0R upload=M9FCIGYESWCZ tag=U2Y25IUUCZ4R type=write size=2089715 2024-05-09T20:17:12.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.147+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~5ZPUWQIC5X4F.0 size=1048576 2024-05-09T20:17:12.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.151+0000 7fdc882e58c0 1 < completed part key=_multipart_P2DNA4DGI54P.2~5ZPUWQIC5X4F.1 size=746982 2024-05-09T20:17:12.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.155+0000 7fdc882e58c0 1 < completed multipart upload key=P2DNA4DGI54P upload=5ZPUWQIC5X4F tag=U1WN5QK3JMTW type=write size=2844134 2024-05-09T20:17:12.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.159+0000 7fdc882e58c0 1 > prepared operation key=J2Q86HKRMAKE tag=BWERD93OQ58J type=del size=3792459 2024-05-09T20:17:12.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.159+0000 7fdc882e58c0 1 < canceled part key=_multipart_T5EL7X0P450L.2~N49W41ZILR3I.0 size=21246 2024-05-09T20:17:12.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.163+0000 7fdc882e58c0 1 < canceled multipart upload key=T5EL7X0P450L upload=N49W41ZILR3I tag=JBL3EA5MY1VK type=write size=1069822 2024-05-09T20:17:12.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.171+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~F93SANUKET0A.0 size=1048576 2024-05-09T20:17:12.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.175+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~F93SANUKET0A.1 size=1048576 2024-05-09T20:17:12.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.175+0000 7fdc882e58c0 1 > prepared part key=_multipart_346HV1XQKCOZ.2~F93SANUKET0A.2 size=508127 2024-05-09T20:17:12.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.175+0000 7fdc882e58c0 1 > prepared multipart upload key=346HV1XQKCOZ upload=F93SANUKET0A tag=OQ8FXPZDTM9U type=write size=3653855 2024-05-09T20:17:12.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.179+0000 7fdc882e58c0 1 < completed delete operation key=NYHR6GTPQYZS tag=QH5KLXQLTDA8 type=del 2024-05-09T20:17:12.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.183+0000 7fdc882e58c0 1 > prepared operation key=G0DX1F257RLD tag=0X20VAL3UZCJ type=del size=3634873 2024-05-09T20:17:12.179 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.187+0000 7fdc882e58c0 -1 < failed to complete operation key=T6T3IOW9VPA0 tag=DM272BWYYNEN type=del size=3192827: (22) Invalid argument 2024-05-09T20:17:12.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.191+0000 7fdc882e58c0 1 > prepared operation key=NBAECV9L85UK tag=A6AWPZ9R2H1Z type=write size=819698 2024-05-09T20:17:12.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.195+0000 7fdc882e58c0 1 < suggested update operation key=5UMJFJRKUPKR tag=7VN42U9BEUE3 type=del size=2149714 2024-05-09T20:17:12.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.199+0000 7fdc882e58c0 1 > prepared operation key=5H2GWM9LQELZ tag=K4Y2YLM8YKR7 type=del size=1669830 2024-05-09T20:17:12.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.207+0000 7fdc882e58c0 1 < suggested update operation key=5H2GWM9LQELZ tag=3F3QUZR40YMV type=write size=530967 2024-05-09T20:17:12.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.215+0000 7fdc882e58c0 1 > prepared part key=_multipart_T5EL7X0P450L.2~8UK0QQSODZYZ.0 size=55852 2024-05-09T20:17:12.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.215+0000 7fdc882e58c0 1 > prepared multipart upload key=T5EL7X0P450L upload=8UK0QQSODZYZ tag=3YTGW9QWUN8O type=write size=1104428 2024-05-09T20:17:12.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.219+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~ZP3HCH3882FQ.0 size=1048576 2024-05-09T20:17:12.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.219+0000 7fdc882e58c0 1 < completed part key=_multipart_T5EL7X0P450L.2~ZP3HCH3882FQ.1 size=1006826 2024-05-09T20:17:12.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.227+0000 7fdc882e58c0 1 < completed multipart upload key=T5EL7X0P450L upload=ZP3HCH3882FQ tag=O2ZJYGZ0O1VZ type=write size=3103978 2024-05-09T20:17:12.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:17:12.231+0000 7fdc882e58c0 1 > prepared operation key=FNB9B18LIA8L tag=M599BSXWCHMG type=write size=758842 2024-05-09T20:17:12.224 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] cls_rgw_stats.simulate (24523 ms) 2024-05-09T20:17:12.224 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 1 test from cls_rgw_stats (24523 ms total) 2024-05-09T20:17:12.224 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:17:12.224 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:17:12.769 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 1 test from 1 test suite ran. (26932 ms total) 2024-05-09T20:17:12.769 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 1 test. 2024-05-09T20:17:12.779 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:17:12.779 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:17:12.835 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2024-05-09T20:17:12.836 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2024-05-09T20:17:12.837 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2024-05-09T20:17:12.901 INFO:tasks.workunit.client.0.smithi063.stdout:Running main() from gmock_main.cc 2024-05-09T20:17:12.902 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 31 tests from 1 test suite. 2024-05-09T20:17:12.902 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:17:14.764 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 31 tests from CmpOmap 2024-05-09T20:17:14.765 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2024-05-09T20:17:14.809 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (45 ms) 2024-05-09T20:17:14.809 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2024-05-09T20:17:14.815 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (6 ms) 2024-05-09T20:17:14.815 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2024-05-09T20:17:14.822 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (7 ms) 2024-05-09T20:17:14.822 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2024-05-09T20:17:14.829 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (6 ms) 2024-05-09T20:17:14.829 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_str 2024-05-09T20:17:14.843 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_str (15 ms) 2024-05-09T20:17:14.843 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2024-05-09T20:17:14.857 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_u64 (14 ms) 2024-05-09T20:17:14.857 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2024-05-09T20:17:14.866 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (9 ms) 2024-05-09T20:17:14.866 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2024-05-09T20:17:14.876 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (10 ms) 2024-05-09T20:17:14.876 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2024-05-09T20:17:15.491 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (614 ms) 2024-05-09T20:17:15.491 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2024-05-09T20:17:15.505 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (15 ms) 2024-05-09T20:17:15.505 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2024-05-09T20:17:15.507 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (2 ms) 2024-05-09T20:17:15.507 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2024-05-09T20:17:15.509 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (2 ms) 2024-05-09T20:17:15.509 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2024-05-09T20:17:15.526 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (17 ms) 2024-05-09T20:17:15.527 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2024-05-09T20:17:15.544 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (18 ms) 2024-05-09T20:17:15.545 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2024-05-09T20:17:15.562 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (18 ms) 2024-05-09T20:17:15.562 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2024-05-09T20:17:15.580 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (18 ms) 2024-05-09T20:17:15.580 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2024-05-09T20:17:15.601 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_str (21 ms) 2024-05-09T20:17:15.602 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2024-05-09T20:17:15.622 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (21 ms) 2024-05-09T20:17:15.622 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2024-05-09T20:17:15.629 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (7 ms) 2024-05-09T20:17:15.629 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2024-05-09T20:17:15.635 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (6 ms) 2024-05-09T20:17:15.635 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2024-05-09T20:17:15.638 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (3 ms) 2024-05-09T20:17:15.639 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2024-05-09T20:17:15.641 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (2 ms) 2024-05-09T20:17:15.641 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2024-05-09T20:17:15.657 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (17 ms) 2024-05-09T20:17:15.658 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2024-05-09T20:17:15.678 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (21 ms) 2024-05-09T20:17:15.679 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2024-05-09T20:17:15.699 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (21 ms) 2024-05-09T20:17:15.699 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2024-05-09T20:17:15.722 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (22 ms) 2024-05-09T20:17:15.722 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2024-05-09T20:17:15.728 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (6 ms) 2024-05-09T20:17:15.728 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2024-05-09T20:17:15.735 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (7 ms) 2024-05-09T20:17:15.735 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2024-05-09T20:17:15.738 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (3 ms) 2024-05-09T20:17:15.738 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2024-05-09T20:17:15.740 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (3 ms) 2024-05-09T20:17:15.740 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2024-05-09T20:17:15.760 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (19 ms) 2024-05-09T20:17:15.760 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 31 tests from CmpOmap (996 ms total) 2024-05-09T20:17:15.760 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:17:15.760 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:17:15.798 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 31 tests from 1 test suite ran. (2896 ms total) 2024-05-09T20:17:15.798 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 31 tests. 2024-05-09T20:17:15.803 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:17:15.803 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:17:15.860 INFO:tasks.workunit:Running workunits matching cls/test_cls_2pc_queue.sh on client.0... 2024-05-09T20:17:15.861 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2024-05-09T20:17:15.862 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2024-05-09T20:17:15.922 INFO:tasks.workunit.client.0.smithi063.stdout:Running main() from gmock_main.cc 2024-05-09T20:17:15.922 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 24 tests from 1 test suite. 2024-05-09T20:17:15.922 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:17:15.922 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 24 tests from TestCls2PCQueue 2024-05-09T20:17:15.922 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2024-05-09T20:17:18.826 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2904 ms) 2024-05-09T20:17:18.826 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2024-05-09T20:17:21.894 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3066 ms) 2024-05-09T20:17:21.894 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.Reserve 2024-05-09T20:17:24.962 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.Reserve (3070 ms) 2024-05-09T20:17:24.962 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2024-05-09T20:17:27.994 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3031 ms) 2024-05-09T20:17:27.994 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.Commit 2024-05-09T20:17:32.153 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.Commit (4161 ms) 2024-05-09T20:17:32.153 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.Stats 2024-05-09T20:17:36.213 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.Stats (4059 ms) 2024-05-09T20:17:36.213 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.UpgradeFromReef 2024-05-09T20:17:40.262 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.UpgradeFromReef (4042 ms) 2024-05-09T20:17:40.263 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.Abort 2024-05-09T20:17:43.304 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.Abort (3050 ms) 2024-05-09T20:17:43.305 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2024-05-09T20:17:46.326 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.ReserveError (3021 ms) 2024-05-09T20:17:46.326 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.CommitError 2024-05-09T20:17:49.367 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.CommitError (3041 ms) 2024-05-09T20:17:49.367 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.AbortError 2024-05-09T20:17:52.430 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.AbortError (3063 ms) 2024-05-09T20:17:52.430 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2024-05-09T20:17:55.472 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3041 ms) 2024-05-09T20:17:55.472 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2024-05-09T20:17:58.476 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3004 ms) 2024-05-09T20:17:58.476 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2024-05-09T20:18:01.559 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3083 ms) 2024-05-09T20:18:01.559 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2024-05-09T20:18:04.582 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3021 ms) 2024-05-09T20:18:04.582 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2024-05-09T20:18:07.622 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3042 ms) 2024-05-09T20:18:07.622 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2024-05-09T20:18:10.674 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3053 ms) 2024-05-09T20:18:10.675 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2024-05-09T20:18:13.684 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.Cleanup (3010 ms) 2024-05-09T20:18:13.684 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2024-05-09T20:18:27.769 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.MultiProducer (14084 ms) 2024-05-09T20:18:27.769 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2024-05-09T20:18:32.784 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (5015 ms) 2024-05-09T20:18:32.784 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2024-05-09T20:18:46.929 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (14145 ms) 2024-05-09T20:18:46.930 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2024-05-09T20:18:53.033 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (6104 ms) 2024-05-09T20:18:53.034 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2024-05-09T20:19:03.102 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (10068 ms) 2024-05-09T20:19:03.102 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2024-05-09T20:19:13.228 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (10126 ms) 2024-05-09T20:19:13.228 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 24 tests from TestCls2PCQueue (117306 ms total) 2024-05-09T20:19:13.228 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:13.228 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:13.228 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 24 tests from 1 test suite ran. (117306 ms total) 2024-05-09T20:19:13.228 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 24 tests. 2024-05-09T20:19:13.233 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:19:13.233 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:19:13.287 INFO:tasks.workunit:Running workunits matching cls/test_cls_user.sh on client.0... 2024-05-09T20:19:13.288 INFO:tasks.workunit:Running workunit cls/test_cls_user.sh... 2024-05-09T20:19:13.288 DEBUG:teuthology.orchestra.run.smithi063:workunit test cls/test_cls_user.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_user.sh 2024-05-09T20:19:13.373 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:13.373 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:13.373 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 4 tests from 1 test suite. 2024-05-09T20:19:13.373 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:15.250 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 4 tests from ClsAccount 2024-05-09T20:19:15.250 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsAccount.add 2024-05-09T20:19:15.271 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsAccount.add (22 ms) 2024-05-09T20:19:15.271 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsAccount.get 2024-05-09T20:19:15.280 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsAccount.get (9 ms) 2024-05-09T20:19:15.280 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsAccount.rm 2024-05-09T20:19:15.293 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsAccount.rm (13 ms) 2024-05-09T20:19:15.293 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] ClsAccount.list 2024-05-09T20:19:15.335 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] ClsAccount.list (42 ms) 2024-05-09T20:19:15.336 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 4 tests from ClsAccount (86 ms total) 2024-05-09T20:19:15.336 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:15.336 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:16.235 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 4 tests from 1 test suite ran. (2862 ms total) 2024-05-09T20:19:16.236 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 4 tests. 2024-05-09T20:19:16.241 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:19:16.241 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:19:16.295 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2024-05-09T20:19:16.296 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2024-05-09T20:19:16.296 DEBUG:teuthology.orchestra.run.smithi063:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2024-05-09T20:19:16.377 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:16.378 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:16.378 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 6 tests from 1 test suite. 2024-05-09T20:19:16.378 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:18.262 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 6 tests from rgw_gc_log 2024-05-09T20:19:18.263 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2024-05-09T20:19:18.305 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] rgw_gc_log.init_existing_queue (43 ms) 2024-05-09T20:19:18.305 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2024-05-09T20:19:18.309 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] rgw_gc_log.init_existing_omap (4 ms) 2024-05-09T20:19:18.309 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2024-05-09T20:19:18.315 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (6 ms) 2024-05-09T20:19:18.315 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2024-05-09T20:19:18.318 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (3 ms) 2024-05-09T20:19:18.318 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2024-05-09T20:19:18.322 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] rgw_gc_log.defer1_after_init (4 ms) 2024-05-09T20:19:18.322 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2024-05-09T20:19:18.325 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] rgw_gc_log.defer2_before_init (3 ms) 2024-05-09T20:19:18.325 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 6 tests from rgw_gc_log (63 ms total) 2024-05-09T20:19:18.325 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:18.325 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:19.270 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 6 tests from 1 test suite ran. (2892 ms total) 2024-05-09T20:19:19.270 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 6 tests. 2024-05-09T20:19:19.275 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:19:19.275 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:19:19.331 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_obj.sh on client.0... 2024-05-09T20:19:19.333 INFO:tasks.workunit:Running workunit rgw/test_rgw_obj.sh... 2024-05-09T20:19:19.333 DEBUG:teuthology.orchestra.run.smithi063:workunit test rgw/test_rgw_obj.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_obj.sh 2024-05-09T20:19:19.403 INFO:tasks.workunit.client.0.smithi063.stdout:Running main() from gmock_main.cc 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 4 tests from 1 test suite. 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 4 tests from TestRGWObj 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestRGWObj.underscore 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns= i= 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns= i= 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.404 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns= i= 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "obj1": { 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.405 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "obj2": { 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.406 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3": { 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.407 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.408 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3-2": { 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.409 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns=ns i= 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns=ns i= 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj 2024-05-09T20:19:19.410 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns=ns i= 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "obj1": { 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.411 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "obj2": { 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.412 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3": { 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.413 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3-2": { 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.414 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.415 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns= i=v1 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns= i=v1 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns= i=v1 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout: "obj1": { 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.416 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.417 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "obj2": { 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.418 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3": { 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.419 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3-2": { 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.420 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.421 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns=ns i=v1 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns=ns i=v1 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: _obj[v1] 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: _obj ns=ns i=v1 2024-05-09T20:19:19.422 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.423 INFO:tasks.workunit.client.0.smithi063.stdout: "obj1": { 2024-05-09T20:19:19.423 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.424 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "obj2": { 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.425 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3": { 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.426 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.427 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3-2": { 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.428 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_obj", 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestRGWObj.underscore (0 ms) 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestRGWObj.no_underscore 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns= i= 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns= i= 2024-05-09T20:19:19.429 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns= i= 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "obj1": { 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.430 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "obj2": { 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.431 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3": { 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.432 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.433 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3-2": { 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.434 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns=ns i= 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns=ns i= 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.435 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns=ns i= 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "obj1": { 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.436 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "obj2": { 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.437 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3": { 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.438 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.439 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3-2": { 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.440 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.441 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns= i=v1 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns= i=v1 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns= i=v1 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout: "obj1": { 2024-05-09T20:19:19.442 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.443 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: "obj2": { 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.444 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3": { 2024-05-09T20:19:19.445 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.446 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3-2": { 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.447 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.448 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns=ns i=v1 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns=ns i=v1 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.449 INFO:tasks.workunit.client.0.smithi063.stdout:parsed: obj[v1] 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout:stripped: obj ns=ns i=v1 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "obj1": { 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.450 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "obj2": { 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.451 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3": { 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.452 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.453 INFO:tasks.workunit.client.0.smithi063.stdout: "obj3-2": { 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.454 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "obj", 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "v1", 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestRGWObj.no_underscore (0 ms) 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestRGWObj.obj_to_raw 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.455 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.456 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.457 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker_myobj", 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.458 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.459 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.460 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.461 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.462 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker_myobj", 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.463 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.464 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.465 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.466 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns_myobj", 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.467 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.468 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.469 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.470 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns_myobj", 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.471 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.472 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.473 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.474 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.475 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst_myobj", 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.476 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.477 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.478 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.479 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst_myobj", 2024-05-09T20:19:19.480 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.481 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.482 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.483 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.484 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.485 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst_myobj", 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.486 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.487 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.488 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.489 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.490 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst_myobj", 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.491 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.492 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.493 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.494 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker___myobj", 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj" 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.495 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.496 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.497 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.498 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.499 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker___myobj", 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj" 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.500 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.501 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.502 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.503 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns__myobj", 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.504 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.505 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.506 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.507 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.508 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns__myobj", 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.509 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.510 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.510 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.510 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.510 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.510 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.510 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.510 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.510 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.511 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.512 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.513 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst__myobj", 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj" 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.514 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.515 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.516 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.517 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.518 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst__myobj", 2024-05-09T20:19:19.519 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj" 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.520 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.521 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.522 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.523 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst__myobj", 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.524 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.525 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.526 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.527 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.528 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst__myobj", 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.529 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.530 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.531 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.532 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker___myobj_", 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj_" 2024-05-09T20:19:19.533 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.534 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.535 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.536 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.537 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker___myobj_", 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj_" 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.538 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.539 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.540 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.541 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns__myobj_", 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.542 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.543 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.544 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.545 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.546 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns__myobj_", 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.547 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.548 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.549 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.550 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.551 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst__myobj_", 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj_" 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.552 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.553 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.554 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.555 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.556 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst__myobj_", 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj_" 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.557 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.558 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.559 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.560 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.561 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": "default-placement.data", 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst__myobj_", 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "test", 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.562 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": "", 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": "" 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.563 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.564 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "obj": { 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.565 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.566 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst__myobj_", 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.567 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestRGWObj.obj_to_raw (1 ms) 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] TestRGWObj.old_to_raw 2024-05-09T20:19:19.568 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker_myobj", 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.569 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker_myobj 2024-05-09T20:19:19.570 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker_myobj", 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.571 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.572 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker_myobj 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns_myobj", 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.573 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__ns_myobj 2024-05-09T20:19:19.574 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns_myobj", 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.575 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.576 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__ns_myobj 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst_myobj", 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.577 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.578 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__:inst_myobj 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst_myobj", 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.579 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.580 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__:inst_myobj 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst_myobj", 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.581 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.582 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__ns:inst_myobj 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst_myobj", 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.583 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.584 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "myobj", 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__ns:inst_myobj 2024-05-09T20:19:19.585 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker___myobj", 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj" 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.586 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.587 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker___myobj 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.588 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker___myobj", 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj" 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.589 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.590 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker___myobj 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns__myobj", 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.591 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.592 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__ns__myobj 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns__myobj", 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.593 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.594 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__ns__myobj 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.595 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst__myobj", 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj" 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.596 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__:inst__myobj 2024-05-09T20:19:19.597 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst__myobj", 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj" 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.598 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.599 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__:inst__myobj 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst__myobj", 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.600 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.601 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__ns:inst__myobj 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst__myobj", 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.602 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj", 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.603 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__ns:inst__myobj 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker___myobj_", 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj_" 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.604 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.605 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker___myobj_ 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker___myobj_", 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj_" 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.606 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.607 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker___myobj_ 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns__myobj_", 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.608 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.609 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__ns__myobj_ 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns__myobj_", 2024-05-09T20:19:19.610 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.611 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "", 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__ns__myobj_ 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst__myobj_", 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj_" 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.612 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.613 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__:inst__myobj_ 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__:inst__myobj_", 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "marker__myobj_" 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.614 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.615 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "" 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__:inst__myobj_ 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj": { 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst__myobj_", 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.616 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj": { 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.617 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout:raw=.data-pool:marker__ns:inst__myobj_ 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: "raw_obj2": { 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: "pool": ".data-pool", 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: "oid": "marker__ns:inst__myobj_", 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: "loc": "" 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.618 INFO:tasks.workunit.client.0.smithi063.stdout: "new_obj2": { 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket": { 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "ebtest", 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "marker": "marker", 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_id": "bucket-id", 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "tenant": "", 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "explicit_placement": { 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "data_pool": ".data-pool", 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "data_extra_pool": "", 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "index_pool": ".index-pool" 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "key": { 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "name": "_myobj_", 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "instance": "inst", 2024-05-09T20:19:19.619 INFO:tasks.workunit.client.0.smithi063.stdout: "ns": "ns" 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout:raw2=.data-pool:marker__ns:inst__myobj_ 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] TestRGWObj.old_to_raw (0 ms) 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 4 tests from TestRGWObj (1 ms total) 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:19.620 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 4 tests from 1 test suite ran. (1 ms total) 2024-05-09T20:19:19.621 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 4 tests. 2024-05-09T20:19:19.621 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:19:19.621 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:19:19.636 INFO:tasks.workunit:Running workunits matching rgw/test_librgw_file.sh on client.0... 2024-05-09T20:19:19.637 INFO:tasks.workunit:Running workunit rgw/test_librgw_file.sh... 2024-05-09T20:19:19.637 DEBUG:teuthology.orchestra.run.smithi063:workunit test rgw/test_librgw_file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_librgw_file.sh 2024-05-09T20:19:19.764 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:19.764 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:19.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 realm 2024-05-09T20:19:19.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:19:19.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:19:19.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.818+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.822+0000 7f538c619980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:19:19.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.822+0000 7f538c619980 20 searching for the correct realm 2024-05-09T20:19:19.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.822+0000 7f538a92e640 0 -- 172.21.15.63:0/1170611361 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f537407ccb0 msgr2=0x7f537405f500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:19:19.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.830+0000 7f538992c640 0 -- 172.21.15.63:0/1170611361 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f537c027040 msgr2=0x7f537c062120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:19:19.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:19:19.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:19:19.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:19:19.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:19:19.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:19:19.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:19:19.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 zone default found 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 4 Realm: () 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:19:19.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 zonegroup default 2024-05-09T20:19:19.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 started sync module instance, tier type = 2024-05-09T20:19:19.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.842+0000 7f538c619980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:19:19.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.858+0000 7f538c619980 20 add_watcher() i=0 2024-05-09T20:19:19.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.858+0000 7f538c619980 20 add_watcher() i=1 2024-05-09T20:19:19.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.858+0000 7f538c619980 20 add_watcher() i=2 2024-05-09T20:19:19.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.858+0000 7f538c619980 20 add_watcher() i=3 2024-05-09T20:19:19.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.858+0000 7f538c619980 20 add_watcher() i=4 2024-05-09T20:19:19.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.858+0000 7f538c619980 20 add_watcher() i=5 2024-05-09T20:19:19.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.858+0000 7f538c619980 20 add_watcher() i=6 2024-05-09T20:19:19.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.862+0000 7f538c619980 20 add_watcher() i=7 2024-05-09T20:19:19.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.862+0000 7f538c619980 2 all 8 watchers are set, enabling cache 2024-05-09T20:19:19.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.862+0000 7f5240fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:19:19.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.862+0000 7f538c619980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:19:19.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.862+0000 7f538c619980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:19:19.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 5 note: GC not initialized 2024-05-09T20:19:19.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f51bb7fe640 20 reqs_thread_entry: start 2024-05-09T20:19:19.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 20 init_complete bucket index max shards: 11 2024-05-09T20:19:19.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 20 Filter name: none 2024-05-09T20:19:19.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f51b97fa640 20 reqs_thread_entry: start 2024-05-09T20:19:19.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-librgw-file : miss 2024-05-09T20:19:19.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x0 2024-05-09T20:19:19.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 adding default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-05-09T20:19:19.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 cache get: name=default.rgw.meta+users.email+librgw@example.com : miss 2024-05-09T20:19:19.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x0 2024-05-09T20:19:19.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 adding default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-05-09T20:19:19.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 cache get: name=default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= : miss 2024-05-09T20:19:19.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 20 rados->read ofs=0 len=0 2024-05-09T20:19:19.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:19.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 cache put: name=default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= info.flags=0x0 2024-05-09T20:19:19.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 adding default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= to cache LRU end 2024-05-09T20:19:19.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 cache get: name=default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= : hit (negative entry) 2024-05-09T20:19:19.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.866+0000 7f538c619980 10 cache get: name=default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= : hit (negative entry) 2024-05-09T20:19:19.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.870+0000 7f538c619980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-05-09T20:19:19.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.870+0000 7f538c619980 10 moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-05-09T20:19:19.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.870+0000 7f538c619980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-librgw-file, ofs0, ns] 2024-05-09T20:19:19.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.870+0000 7f526dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791297 cookie 94689063277648 notifier 5520 bl.length()=697 2024-05-09T20:19:19.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.870+0000 7f526dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-librgw-file info.flags=0x17 2024-05-09T20:19:19.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.870+0000 7f526dffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-librgw-file to cache LRU end 2024-05-09T20:19:19.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.874+0000 7f538c619980 10 cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-05-09T20:19:19.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.874+0000 7f538c619980 10 moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-05-09T20:19:19.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.874+0000 7f538c619980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:librgw@example.com, ofs0, ns] 2024-05-09T20:19:19.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.874+0000 7f526d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791296 cookie 94689063273456 notifier 5520 bl.length()=194 2024-05-09T20:19:19.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.874+0000 7f526d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+librgw@example.com info.flags=0x7 2024-05-09T20:19:19.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.874+0000 7f526d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+librgw@example.com to cache LRU end 2024-05-09T20:19:19.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.878+0000 7f538c619980 10 cache put: name=default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= info.flags=0x7 2024-05-09T20:19:19.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.878+0000 7f538c619980 10 moving default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= to cache LRU end 2024-05-09T20:19:19.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.878+0000 7f538c619980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.keys:0qG7TK40C04rozekhVzplx/KUio=, ofs0, ns] 2024-05-09T20:19:19.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.878+0000 7f526dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791296 cookie 94689063290832 notifier 5520 bl.length()=203 2024-05-09T20:19:19.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.878+0000 7f526dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= info.flags=0x7 2024-05-09T20:19:19.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.878+0000 7f526dffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+0qG7TK40C04rozekhVzplx/KUio= to cache LRU end 2024-05-09T20:19:19.874 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "user_id": "ceph-test-librgw-file", 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "display_name": "librgw test user", 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "email": "librgw@example.com", 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "suspended": 0, 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "subusers": [], 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "keys": [ 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: { 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "user": "ceph-test-librgw-file", 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "access_key": "0qG7TK40C04rozekhVzplx/KUio=", 2024-05-09T20:19:19.875 INFO:tasks.workunit.client.0.smithi063.stdout: "secret_key": "f9p6uf25E0Bvfg32RlnyQliQi/Pc1X+fVqz0w4BNVynQSlraGIIJFQ==", 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "active": true, 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "create_date": "2024-05-09T20:19:19.872057Z" 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: ], 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "swift_keys": [], 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "caps": [], 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "default_placement": "", 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "placement_tags": [], 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_quota": { 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "enabled": false, 2024-05-09T20:19:19.876 INFO:tasks.workunit.client.0.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size": -1, 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "max_objects": -1 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "user_quota": { 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "enabled": false, 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size": -1, 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:19:19.877 INFO:tasks.workunit.client.0.smithi063.stdout: "max_objects": -1 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: "type": "rgw", 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: "account_id": "", 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: "path": "/", 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: "create_date": "2024-05-09T20:19:19.872037Z", 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: "tags": [], 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: "group_ids": [] 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:19.878 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:19.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.882+0000 7f5240fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:19:19.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.886+0000 7f538c619980 20 remove_watcher() i=0 2024-05-09T20:19:19.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.886+0000 7f538c619980 2 removed watcher, disabling cache 2024-05-09T20:19:19.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.890+0000 7f538c619980 20 remove_watcher() i=1 2024-05-09T20:19:19.884 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.890+0000 7f538c619980 20 remove_watcher() i=2 2024-05-09T20:19:19.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.894+0000 7f538c619980 20 remove_watcher() i=3 2024-05-09T20:19:19.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.894+0000 7f538c619980 20 remove_watcher() i=4 2024-05-09T20:19:19.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.898+0000 7f538c619980 20 remove_watcher() i=5 2024-05-09T20:19:19.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.898+0000 7f538c619980 20 remove_watcher() i=6 2024-05-09T20:19:19.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:19.898+0000 7f538c619980 20 remove_watcher() i=7 2024-05-09T20:19:19.905 INFO:tasks.workunit.client.0.smithi063.stdout:phase 1.1 2024-05-09T20:19:19.922 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 24 tests from 1 test suite. 2024-05-09T20:19:19.922 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:19.922 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 24 tests from LibRGW 2024-05-09T20:19:19.923 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:19.934 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:19.934 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:20.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:20.346+0000 7f59efe82a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:20.511 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (589 ms) 2024-05-09T20:19:20.511 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-05-09T20:19:20.513 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-05-09T20:19:20.513 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:20.513 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-05-09T20:19:20.513 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-05-09T20:19:20.535 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/baz/quux 2024-05-09T20:19:20.542 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/f1 2024-05-09T20:19:20.547 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/f2 2024-05-09T20:19:20.552 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/f1 2024-05-09T20:19:20.558 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/d1/ 2024-05-09T20:19:20.564 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/baz/hungry 2024-05-09T20:19:20.570 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/baz/hungry/ 2024-05-09T20:19:20.575 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/baz/momma 2024-05-09T20:19:20.581 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/baz/bear/ 2024-05-09T20:19:20.587 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/baz/sasquatch 2024-05-09T20:19:20.594 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/baz/sasquatch/ 2024-05-09T20:19:20.601 INFO:tasks.workunit.client.0.smithi063.stdout:creating: nfsroot:foo/bar/baz/frobozz 2024-05-09T20:19:20.607 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETUP_HIER1 (94 ms) 2024-05-09T20:19:20.607 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-05-09T20:19:20.846 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETUP_DIRS1 (239 ms) 2024-05-09T20:19:20.846 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETATTR 2024-05-09T20:19:20.846 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-05-09T20:19:20.846 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-05-09T20:19:20.891 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (45 ms) 2024-05-09T20:19:20.891 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-05-09T20:19:20.990 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (98 ms) 2024-05-09T20:19:20.990 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-05-09T20:19:20.990 INFO:tasks.workunit.client.0.smithi063.stdout:rename file rfile_0 to rfile_r0 (bucket brename0) 2024-05-09T20:19:21.006 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (17 ms) 2024-05-09T20:19:21.006 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-05-09T20:19:21.006 INFO:tasks.workunit.client.0.smithi063.stdout:rename file rfile_1 (bucket brename0) to rfile_rhilldog (bucket brename1) 2024-05-09T20:19:21.022 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (16 ms) 2024-05-09T20:19:21.022 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-05-09T20:19:21.022 INFO:tasks.workunit.client.0.smithi063.stdout:scanning objects in /bdirs1/dir_0 2024-05-09T20:19:21.022 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:21.022 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-09T20:19:21.022 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout:scanning objects in /bdirs1/dir_1 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout:scanning objects in /bdirs1/dir_2 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-05-09T20:19:21.023 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-05-09T20:19:21.024 INFO:tasks.workunit.client.0.smithi063.stdout:read back objects in /bdirs1/dir_0 2024-05-09T20:19:21.024 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_0/sfile_0 2024-05-09T20:19:21.027 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-09T20:19:21.027 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_0/sfile_1 2024-05-09T20:19:21.031 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-09T20:19:21.031 INFO:tasks.workunit.client.0.smithi063.stdout:read back objects in /bdirs1/dir_1 2024-05-09T20:19:21.031 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_1/sfile_0 2024-05-09T20:19:21.035 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-09T20:19:21.035 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_1/sfile_1 2024-05-09T20:19:21.038 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-09T20:19:21.038 INFO:tasks.workunit.client.0.smithi063.stdout:read back objects in /bdirs1/dir_2 2024-05-09T20:19:21.038 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_2/sfile_0 2024-05-09T20:19:21.041 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-09T20:19:21.041 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_2/sfile_1 2024-05-09T20:19:21.045 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-09T20:19:21.046 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READ_DIRS1 (24 ms) 2024-05-09T20:19:21.046 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READF_SETUP1 (189 ms) 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_0 type: DIR refs: 2 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_0 type: FILE refs: 2 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_1 type: DIR refs: 2 2024-05-09T20:19:21.236 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_1 type: FILE refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_0 type: DIR refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_0 type: FILE refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_1 type: DIR refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_1 type: FILE refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_0 type: DIR refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_0 type: FILE refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_1 type: DIR refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_1 type: FILE refs: 2 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (1 ms) 2024-05-09T20:19:21.237 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.HIER1 2024-05-09T20:19:21.255 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/f2 (f2): FILE 2024-05-09T20:19:21.256 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/f1 (f1): FILE 2024-05-09T20:19:21.262 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-05-09T20:19:21.266 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-05-09T20:19:21.272 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-05-09T20:19:21.273 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-05-09T20:19:21.273 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-05-09T20:19:21.274 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-05-09T20:19:21.274 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-05-09T20:19:21.275 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-05-09T20:19:21.276 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-05-09T20:19:21.282 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-05-09T20:19:21.282 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-05-09T20:19:21.282 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar (bar): DIR 2024-05-09T20:19:21.282 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo (foo): DIR 2024-05-09T20:19:21.282 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot (nfsroot): DIR 2024-05-09T20:19:21.282 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.HIER1 (44 ms) 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:21.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:21.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:21.286+0000 7f59efe82a80 -1 shutting down 2024-05-09T20:19:21.284 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:21.290+0000 7f58cb3d7640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 53781, si_uid : 1000, si_addr0x3e80000d215, si_status0 2024-05-09T20:19:21.459 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (178 ms) 2024-05-09T20:19:21.459 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 24 tests from LibRGW (1536 ms total) 2024-05-09T20:19:21.459 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:21.459 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:21.459 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 24 tests from 1 test suite ran. (1536 ms total) 2024-05-09T20:19:21.459 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 24 tests. 2024-05-09T20:19:21.470 INFO:tasks.workunit.client.0.smithi063.stdout:phase 1.2 2024-05-09T20:19:21.493 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 24 tests from 1 test suite. 2024-05-09T20:19:21.493 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:21.493 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 24 tests from LibRGW 2024-05-09T20:19:21.493 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:21.503 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:21.503 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:22.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:22.130+0000 7fa4fe73ba80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:22.262 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (769 ms) 2024-05-09T20:19:22.262 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT_NOROOT (1 ms) 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETUP_HIER1 (0 ms) 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETUP_DIRS1 (0 ms) 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETATTR 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-05-09T20:19:22.264 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READ_DIRS1 (0 ms) 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-05-09T20:19:22.265 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.HIER1 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.HIER1 (0 ms) 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-09T20:19:22.266 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-09T20:19:22.267 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:22.267 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:22.267 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:22.267 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:22.267 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:22.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:22.270+0000 7fa4fe73ba80 -1 shutting down 2024-05-09T20:19:22.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:22.274+0000 7fa3d9c90640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 54425, si_uid : 1000, si_addr0x3e80000d499, si_status0 2024-05-09T20:19:22.404 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (141 ms) 2024-05-09T20:19:22.404 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 24 tests from LibRGW (911 ms total) 2024-05-09T20:19:22.405 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:22.405 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:22.405 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 24 tests from 1 test suite ran. (911 ms total) 2024-05-09T20:19:22.405 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 24 tests. 2024-05-09T20:19:22.420 INFO:tasks.workunit.client.0.smithi063.stdout:phase 1.3 2024-05-09T20:19:22.444 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 24 tests from 1 test suite. 2024-05-09T20:19:22.444 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:22.444 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 24 tests from LibRGW 2024-05-09T20:19:22.444 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:22.457 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:22.457 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:22.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:22.874+0000 7f8c8a662a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:22.998 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (553 ms) 2024-05-09T20:19:22.998 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT_NOROOT 2024-05-09T20:19:22.999 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT_NOROOT (2 ms) 2024-05-09T20:19:22.999 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:22.999 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (0 ms) 2024-05-09T20:19:22.999 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETUP_HIER1 2024-05-09T20:19:23.007 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETUP_HIER1 (8 ms) 2024-05-09T20:19:23.008 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETUP_DIRS1 2024-05-09T20:19:23.134 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETUP_DIRS1 (125 ms) 2024-05-09T20:19:23.134 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SETATTR 2024-05-09T20:19:23.134 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SETATTR (0 ms) 2024-05-09T20:19:23.134 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_CREATE_DIRS1 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_CREATE_DIRS1 (0 ms) 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_SETUP_RENAME1 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_SETUP_RENAME1 (0 ms) 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_INTRABUCKET_RENAME1 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_INTRABUCKET_RENAME1 (0 ms) 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RGW_CROSSBUCKET_RENAME1 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RGW_CROSSBUCKET_RENAME1 (0 ms) 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.GETATTR_DIRS1 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout:scanning objects in /bdirs1/dir_0 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_0/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:23.135 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_0/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_0/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_0/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout:scanning objects in /bdirs1/dir_1 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_1/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_1/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_1/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_1/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout:scanning objects in /bdirs1/dir_2 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_2/sdir_0 (sdir_0): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_2/sfile_0 (sfile_0): FILE mode: 33412 size: 16 2024-05-09T20:19:23.136 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_2/sdir_1 (sdir_1): DIR mode: 17139 nlinks: 1 2024-05-09T20:19:23.137 INFO:tasks.workunit.client.0.smithi063.stdout: /bdirs1/dir_2/sfile_1 (sfile_1): FILE mode: 33412 size: 16 2024-05-09T20:19:23.137 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.GETATTR_DIRS1 (0 ms) 2024-05-09T20:19:23.137 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READ_DIRS1 2024-05-09T20:19:23.137 INFO:tasks.workunit.client.0.smithi063.stdout:read back objects in /bdirs1/dir_0 2024-05-09T20:19:23.137 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_0/sfile_0 2024-05-09T20:19:23.138 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-09T20:19:23.139 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_0/sfile_1 2024-05-09T20:19:23.142 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-09T20:19:23.142 INFO:tasks.workunit.client.0.smithi063.stdout:read back objects in /bdirs1/dir_1 2024-05-09T20:19:23.142 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_1/sfile_0 2024-05-09T20:19:23.144 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-09T20:19:23.144 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_1/sfile_1 2024-05-09T20:19:23.148 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-09T20:19:23.148 INFO:tasks.workunit.client.0.smithi063.stdout:read back objects in /bdirs1/dir_2 2024-05-09T20:19:23.148 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_2/sfile_0 2024-05-09T20:19:23.149 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_0 : "data for sfile_0" 2024-05-09T20:19:23.149 INFO:tasks.workunit.client.0.smithi063.stdout:reading 0,256 dir_2/sfile_1 2024-05-09T20:19:23.154 INFO:tasks.workunit.client.0.smithi063.stdout: read back from sfile_1 : "data for sfile_1" 2024-05-09T20:19:23.154 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READ_DIRS1 (19 ms) 2024-05-09T20:19:23.154 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READF_SETUP1 2024-05-09T20:19:23.154 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READF_SETUP1 (0 ms) 2024-05-09T20:19:23.154 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READF_DIRS1 2024-05-09T20:19:23.154 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READF_DIRS1 (0 ms) 2024-05-09T20:19:23.154 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.WRITEF_DIRS1 2024-05-09T20:19:23.154 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.WRITEF_DIRS1 (0 ms) 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.RELEASE_DIRS1 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_0 type: DIR refs: 2 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_0 type: FILE refs: 2 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_1 type: DIR refs: 2 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_1 type: FILE refs: 2 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_0 type: DIR refs: 2 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_0 type: FILE refs: 2 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_1 type: DIR refs: 2 2024-05-09T20:19:23.155 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_1 type: FILE refs: 2 2024-05-09T20:19:23.156 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_0 type: DIR refs: 2 2024-05-09T20:19:23.156 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_0 type: FILE refs: 2 2024-05-09T20:19:23.156 INFO:tasks.workunit.client.0.smithi063.stdout:release sdir_1 type: DIR refs: 2 2024-05-09T20:19:23.156 INFO:tasks.workunit.client.0.smithi063.stdout:release sfile_1 type: FILE refs: 2 2024-05-09T20:19:23.156 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.RELEASE_DIRS1 (0 ms) 2024-05-09T20:19:23.156 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.HIER1 2024-05-09T20:19:23.177 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/f2 (f2): FILE 2024-05-09T20:19:23.178 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/f1 (f1): FILE 2024-05-09T20:19:23.185 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/f1 (f1): FILE 2024-05-09T20:19:23.193 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/d1 (d1): DIR 2024-05-09T20:19:23.198 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-05-09T20:19:23.199 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/sasquatch (sasquatch): FILE 2024-05-09T20:19:23.201 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/quux (quux): FILE 2024-05-09T20:19:23.202 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/momma (momma): FILE 2024-05-09T20:19:23.204 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-05-09T20:19:23.204 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/hungry (hungry): FILE 2024-05-09T20:19:23.205 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/frobozz (frobozz): FILE 2024-05-09T20:19:23.209 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz/bear (bear): DIR 2024-05-09T20:19:23.209 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar/baz (baz): DIR 2024-05-09T20:19:23.209 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo/bar (bar): DIR 2024-05-09T20:19:23.210 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot/foo (foo): DIR 2024-05-09T20:19:23.210 INFO:tasks.workunit.client.0.smithi063.stdout:/nfsroot (nfsroot): DIR 2024-05-09T20:19:23.210 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.HIER1 (55 ms) 2024-05-09T20:19:23.210 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-09T20:19:23.210 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (0 ms) 2024-05-09T20:19:23.210 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-09T20:19:23.210 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-05-09T20:19:23.210 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-09T20:19:23.211 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-05-09T20:19:23.211 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-09T20:19:23.211 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-09T20:19:23.211 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:23.211 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:23.211 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:23.211 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:23.211 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:23.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:23.214+0000 7f8c8a662a80 -1 shutting down 2024-05-09T20:19:23.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:23.218+0000 7f8b65bb7640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 55069, si_uid : 1000, si_addr0x3e80000d71d, si_status0 2024-05-09T20:19:23.362 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (154 ms) 2024-05-09T20:19:23.362 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 24 tests from LibRGW (917 ms total) 2024-05-09T20:19:23.362 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:23.362 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:23.362 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 24 tests from 1 test suite ran. (917 ms total) 2024-05-09T20:19:23.362 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 24 tests. 2024-05-09T20:19:23.377 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2.1 2024-05-09T20:19:23.399 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 9 tests from 1 test suite. 2024-05-09T20:19:23.399 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:23.400 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 9 tests from LibRGW 2024-05-09T20:19:23.400 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:23.411 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:23.411 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:24.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:24.018+0000 7f178c791a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:24.170 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (770 ms) 2024-05-09T20:19:24.170 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:24.171 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-05-09T20:19:24.172 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-09T20:19:24.191 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKET (20 ms) 2024-05-09T20:19:24.191 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-09T20:19:24.191 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-05-09T20:19:24.192 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-05-09T20:19:24.211 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave0 2024-05-09T20:19:24.231 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave1 2024-05-09T20:19:24.248 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave2 2024-05-09T20:19:24.271 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave3 2024-05-09T20:19:24.292 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave4 2024-05-09T20:19:24.312 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave5 2024-05-09T20:19:24.334 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave6 2024-05-09T20:19:24.357 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave7 2024-05-09T20:19:24.378 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave8 2024-05-09T20:19:24.399 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave9 2024-05-09T20:19:24.399 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (207 ms) 2024-05-09T20:19:24.399 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-05-09T20:19:25.230 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (832 ms) 2024-05-09T20:19:25.230 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:25.230 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:25.230 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:25.231 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:25.231 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:25.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:25.238+0000 7f178c791a80 -1 shutting down 2024-05-09T20:19:25.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:25.238+0000 7f1667ce6640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 55713, si_uid : 1000, si_addr0x3e80000d9a1, si_status0 2024-05-09T20:19:25.431 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (201 ms) 2024-05-09T20:19:25.431 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 9 tests from LibRGW (2032 ms total) 2024-05-09T20:19:25.432 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:25.432 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:25.432 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 9 tests from 1 test suite ran. (2032 ms total) 2024-05-09T20:19:25.432 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 9 tests. 2024-05-09T20:19:25.444 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2.2 2024-05-09T20:19:25.467 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 9 tests from 1 test suite. 2024-05-09T20:19:25.467 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:25.467 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 9 tests from LibRGW 2024-05-09T20:19:25.467 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:25.478 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:25.478 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:25.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:25.654+0000 7efd3c469a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:25.788 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (322 ms) 2024-05-09T20:19:25.789 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:25.790 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-05-09T20:19:25.790 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-09T20:19:25.790 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-05-09T20:19:25.790 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-09T20:19:25.850 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_BUCKET (60 ms) 2024-05-09T20:19:25.851 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKET_MULTI 2024-05-09T20:19:25.872 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave0 2024-05-09T20:19:25.895 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave1 2024-05-09T20:19:25.917 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave2 2024-05-09T20:19:25.938 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave3 2024-05-09T20:19:25.960 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave4 2024-05-09T20:19:25.982 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave5 2024-05-09T20:19:26.008 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave6 2024-05-09T20:19:26.028 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave7 2024-05-09T20:19:26.049 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave8 2024-05-09T20:19:26.073 INFO:tasks.workunit.client.0.smithi063.stdout:created: sorrydave9 2024-05-09T20:19:26.073 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKET_MULTI (223 ms) 2024-05-09T20:19:26.073 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_BUCKET_MULTI 2024-05-09T20:19:26.920 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_BUCKET_MULTI (846 ms) 2024-05-09T20:19:26.920 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:26.920 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:26.920 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:26.920 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:26.920 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:26.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:26.926+0000 7efd3c469a80 -1 shutting down 2024-05-09T20:19:26.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:26.930+0000 7efc179be640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 56357, si_uid : 1000, si_addr0x3e80000dc25, si_status0 2024-05-09T20:19:27.055 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (136 ms) 2024-05-09T20:19:27.056 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 9 tests from LibRGW (1589 ms total) 2024-05-09T20:19:27.056 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:27.056 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:27.056 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 9 tests from 1 test suite ran. (1589 ms total) 2024-05-09T20:19:27.056 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 9 tests. 2024-05-09T20:19:27.068 INFO:tasks.workunit.client.0.smithi063.stdout:phase 3.1 2024-05-09T20:19:27.412 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 13 tests from 1 test suite. 2024-05-09T20:19:27.413 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:27.413 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 13 tests from LibRGW 2024-05-09T20:19:27.413 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:27.422 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:27.422 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:27.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:27.590+0000 7f026ea42a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:27.721 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (309 ms) 2024-05-09T20:19:27.721 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:27.722 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-09T20:19:27.722 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-09T20:19:27.741 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKET (19 ms) 2024-05-09T20:19:27.741 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-05-09T20:19:27.747 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (6 ms) 2024-05-09T20:19:27.747 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-05-09T20:19:27.750 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (3 ms) 2024-05-09T20:19:27.750 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LARGE1 2024-05-09T20:19:27.838 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LARGE1 (88 ms) 2024-05-09T20:19:27.839 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LARGE2 2024-05-09T20:19:27.903 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LARGE2 (65 ms) 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:27.904 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:27.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:27.910+0000 7f026ea42a80 -1 shutting down 2024-05-09T20:19:27.907 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:27.914+0000 7f0149f97640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57001, si_uid : 1000, si_addr0x3e80000dea9, si_status0 2024-05-09T20:19:28.089 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (186 ms) 2024-05-09T20:19:28.089 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 13 tests from LibRGW (677 ms total) 2024-05-09T20:19:28.089 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:28.089 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:28.089 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 13 tests from 1 test suite ran. (677 ms total) 2024-05-09T20:19:28.089 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 13 tests. 2024-05-09T20:19:28.104 INFO:tasks.workunit.client.0.smithi063.stdout:phase 3.2 2024-05-09T20:19:28.453 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 13 tests from 1 test suite. 2024-05-09T20:19:28.453 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:28.453 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 13 tests from LibRGW 2024-05-09T20:19:28.453 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:28.462 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:28.462 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:28.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:28.630+0000 7fdaf17b2a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:28.744 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (292 ms) 2024-05-09T20:19:28.745 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:28.746 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-09T20:19:28.746 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-09T20:19:28.746 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-05-09T20:19:28.746 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-05-09T20:19:28.754 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (8 ms) 2024-05-09T20:19:28.754 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-05-09T20:19:28.761 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (7 ms) 2024-05-09T20:19:28.761 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LARGE1 2024-05-09T20:19:29.315 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LARGE1 (555 ms) 2024-05-09T20:19:29.315 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LARGE2 2024-05-09T20:19:29.315 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LARGE2 (0 ms) 2024-05-09T20:19:29.315 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-05-09T20:19:29.315 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-05-09T20:19:29.315 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-05-09T20:19:29.326 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_OBJECT (11 ms) 2024-05-09T20:19:29.327 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-09T20:19:29.400 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_BUCKET (74 ms) 2024-05-09T20:19:29.400 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:29.400 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:29.400 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:29.400 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:29.400 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:29.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:29.406+0000 7fdaf17b2a80 -1 shutting down 2024-05-09T20:19:29.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:29.410+0000 7fd9ccd07640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 57645, si_uid : 1000, si_addr0x3e80000e12d, si_status0 2024-05-09T20:19:29.505 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (105 ms) 2024-05-09T20:19:29.505 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 13 tests from LibRGW (1053 ms total) 2024-05-09T20:19:29.505 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:29.505 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:29.505 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 13 tests from 1 test suite ran. (1053 ms total) 2024-05-09T20:19:29.505 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 13 tests. 2024-05-09T20:19:29.519 INFO:tasks.workunit.client.0.smithi063.stdout:phase 4.1 2024-05-09T20:19:29.542 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 11 tests from 1 test suite. 2024-05-09T20:19:29.542 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:29.542 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 11 tests from LibRGW 2024-05-09T20:19:29.542 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.TVAR 2024-05-09T20:19:29.542 INFO:tasks.workunit.client.0.smithi063.stdout:read i1: 64001 2024-05-09T20:19:29.543 INFO:tasks.workunit.client.0.smithi063.stdout:read s1: blunderbuss 2024-05-09T20:19:29.543 INFO:tasks.workunit.client.0.smithi063.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-05-09T20:19:29.543 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:29.556 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:29.556 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:29.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:29.974+0000 7fa0c75fca80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:30.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (740 ms) 2024-05-09T20:19:30.283 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:30.285 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-05-09T20:19:30.286 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-09T20:19:30.320 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (35 ms) 2024-05-09T20:19:30.320 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-09T20:19:31.578 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_0 2024-05-09T20:19:31.578 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_1 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_2 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_3 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_4 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_5 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_6 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_7 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_8 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_9 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_10 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_11 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_12 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_13 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_14 2024-05-09T20:19:31.579 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_15 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_16 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_17 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_18 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_19 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_20 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_21 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_22 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_23 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_24 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_25 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_26 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_27 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_28 2024-05-09T20:19:31.580 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_29 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_30 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_31 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_32 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_33 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_34 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_35 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_36 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_37 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_38 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_39 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_40 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_41 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_42 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_43 2024-05-09T20:19:31.581 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_44 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_45 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_46 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_47 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_48 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_49 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_50 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_51 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_52 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_53 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_54 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_55 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_56 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_57 2024-05-09T20:19:31.582 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_58 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_59 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_60 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_61 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_62 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_63 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_64 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_65 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_66 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_67 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_68 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_69 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_70 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_71 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_72 2024-05-09T20:19:31.583 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_73 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_74 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_75 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_76 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_77 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_78 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_79 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_80 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_81 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_82 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_83 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_84 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_85 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_86 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_87 2024-05-09T20:19:31.584 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_88 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_89 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_90 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_91 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_92 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_93 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_94 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_95 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_96 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_97 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_98 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:created: dmarker:f_99 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (1259 ms) 2024-05-09T20:19:31.585 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=. flags=4 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=.. flags=4 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-05-09T20:19:31.587 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-05-09T20:19:31.588 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-05-09T20:19:31.589 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-05-09T20:19:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-05-09T20:19:31.591 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-05-09T20:19:31.592 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-05-09T20:19:31.593 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-05-09T20:19:31.594 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_READDIR (9 ms) 2024-05-09T20:19:31.595 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-05-09T20:19:31.596 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-05-09T20:19:31.596 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-05-09T20:19:31.596 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-05-09T20:19:31.596 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-05-09T20:19:31.596 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-05-09T20:19:31.596 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-05-09T20:19:31.596 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-05-09T20:19:31.597 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-05-09T20:19:31.598 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-05-09T20:19:31.599 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-05-09T20:19:31.600 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-05-09T20:19:31.601 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-05-09T20:19:31.602 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER2_READDIR (7 ms) 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:31.603 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:31.604 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (1 ms) 2024-05-09T20:19:31.604 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:31.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:31.602+0000 7fa0c75fca80 -1 shutting down 2024-05-09T20:19:31.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:31.606+0000 7f9fa2b51640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58289, si_uid : 1000, si_addr0x3e80000e3b1, si_status0 2024-05-09T20:19:31.778 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (183 ms) 2024-05-09T20:19:31.778 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 11 tests from LibRGW (2236 ms total) 2024-05-09T20:19:31.778 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:31.778 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:31.778 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 11 tests from 1 test suite ran. (2236 ms total) 2024-05-09T20:19:31.779 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 11 tests. 2024-05-09T20:19:31.796 INFO:tasks.workunit.client.0.smithi063.stdout:phase 4.2 2024-05-09T20:19:31.820 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 11 tests from 1 test suite. 2024-05-09T20:19:31.820 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:31.820 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 11 tests from LibRGW 2024-05-09T20:19:31.820 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.TVAR 2024-05-09T20:19:31.821 INFO:tasks.workunit.client.0.smithi063.stdout:read i1: 64001 2024-05-09T20:19:31.821 INFO:tasks.workunit.client.0.smithi063.stdout:read s1: blunderbuss 2024-05-09T20:19:31.821 INFO:tasks.workunit.client.0.smithi063.stdout:read s3: [ OK ] LibRGW.TVAR (0 ms) 2024-05-09T20:19:31.821 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:31.831 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:31.831 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:32.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:32.654+0000 7f5ee3e58a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:32.798 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (978 ms) 2024-05-09T20:19:32.798 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:32.800 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-05-09T20:19:32.800 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_BUCKET 2024-05-09T20:19:32.817 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_BUCKET (16 ms) 2024-05-09T20:19:32.817 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_SETUP_OBJECTS 2024-05-09T20:19:32.817 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_SETUP_OBJECTS (0 ms) 2024-05-09T20:19:32.817 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_READDIR 2024-05-09T20:19:32.817 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_READDIR (0 ms) 2024-05-09T20:19:32.817 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER2_READDIR 2024-05-09T20:19:32.820 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_0 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_1 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_10 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_11 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_12 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_13 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_14 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_15 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_16 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_17 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_18 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_19 flags=8 2024-05-09T20:19:32.821 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_2 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_20 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_21 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_22 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_23 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_24 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_25 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_26 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_27 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_28 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_29 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_3 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_30 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_31 flags=8 2024-05-09T20:19:32.822 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_32 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_33 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_34 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_35 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_36 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_37 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_38 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_39 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_4 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_40 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_41 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_42 flags=8 2024-05-09T20:19:32.823 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_43 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_44 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_45 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_46 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_47 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_48 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_49 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_5 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_50 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_51 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_52 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_53 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_54 flags=8 2024-05-09T20:19:32.824 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_55 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_56 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_57 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_58 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_59 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_6 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_60 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_61 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_62 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_63 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_64 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_65 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_66 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_67 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_68 flags=8 2024-05-09T20:19:32.825 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_69 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_7 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_70 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_71 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_72 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_73 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_74 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_75 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_76 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_77 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_78 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_79 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_8 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_80 flags=8 2024-05-09T20:19:32.826 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_81 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_82 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_83 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_84 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_85 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_86 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_87 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_88 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_89 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_9 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_90 flags=8 2024-05-09T20:19:32.827 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_91 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_92 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_93 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_94 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_95 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_96 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_97 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_98 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:r2_cb bucket=dmarker dir=nfs_marker iv count=0 called back name=f_99 flags=8 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER2_READDIR (4 ms) 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MARKER1_OBJ_CLEANUP 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MARKER1_OBJ_CLEANUP (0 ms) 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:32.828 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:32.829 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:32.829 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:32.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:32.826+0000 7f5ee3e58a80 -1 shutting down 2024-05-09T20:19:32.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:32.830+0000 7f5dbf3ad640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 58933, si_uid : 1000, si_addr0x3e80000e635, si_status0 2024-05-09T20:19:32.951 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (131 ms) 2024-05-09T20:19:32.951 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 11 tests from LibRGW (1131 ms total) 2024-05-09T20:19:32.951 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:32.951 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:32.951 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 11 tests from 1 test suite ran. (1131 ms total) 2024-05-09T20:19:32.952 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 11 tests. 2024-05-09T20:19:32.965 INFO:tasks.workunit.client.0.smithi063.stdout:phase 5.1 2024-05-09T20:19:32.999 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 19 tests from 1 test suite. 2024-05-09T20:19:32.999 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:32.999 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 19 tests from LibRGW 2024-05-09T20:19:33.000 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:33.011 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:33.011 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:33.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:33.418+0000 7f9481691a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:33.564 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (565 ms) 2024-05-09T20:19:33.564 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:33.565 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-09T20:19:33.565 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-09T20:19:33.585 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKET (20 ms) 2024-05-09T20:19:33.586 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-05-09T20:19:33.593 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (8 ms) 2024-05-09T20:19:33.593 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-05-09T20:19:33.593 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-05-09T20:19:33.593 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-05-09T20:19:33.597 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (4 ms) 2024-05-09T20:19:33.597 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-05-09T20:19:33.597 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-05-09T20:19:33.597 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-05-09T20:19:33.604 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.PUT_OBJECT (7 ms) 2024-05-09T20:19:33.604 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-05-09T20:19:33.611 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.GET_OBJECT (6 ms) 2024-05-09T20:19:33.611 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-05-09T20:19:33.611 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-05-09T20:19:33.611 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-05-09T20:19:33.611 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-05-09T20:19:33.611 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.WRITEV 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READV 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READV (0 ms) 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_OBJECT (0 ms) 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_BUCKET (0 ms) 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:33.612 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:33.613 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:33.613 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:33.613 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:33.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:33.618+0000 7f9481691a80 -1 shutting down 2024-05-09T20:19:33.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:33.618+0000 7f935cbe6640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 59577, si_uid : 1000, si_addr0x3e80000e8b9, si_status0 2024-05-09T20:19:33.704 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (94 ms) 2024-05-09T20:19:33.704 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 19 tests from LibRGW (705 ms total) 2024-05-09T20:19:33.705 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:33.705 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:33.705 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 19 tests from 1 test suite ran. (705 ms total) 2024-05-09T20:19:33.705 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 19 tests. 2024-05-09T20:19:33.712 INFO:tasks.workunit.client.0.smithi063.stdout:phase 5.2 2024-05-09T20:19:33.741 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 19 tests from 1 test suite. 2024-05-09T20:19:33.741 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:33.741 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 19 tests from LibRGW 2024-05-09T20:19:33.741 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:33.749 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:33.749 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:34.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:34.158+0000 7fe91b5f1a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:34.285 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (545 ms) 2024-05-09T20:19:34.285 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:34.286 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (1 ms) 2024-05-09T20:19:34.286 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKET 2024-05-09T20:19:34.286 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKET (0 ms) 2024-05-09T20:19:34.286 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_BUCKET 2024-05-09T20:19:34.296 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_BUCKET (10 ms) 2024-05-09T20:19:34.296 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LIST_OBJECTS 2024-05-09T20:19:34.296 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LIST_OBJECTS (0 ms) 2024-05-09T20:19:34.296 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_OBJECT 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_OBJECT (0 ms) 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.OBJ_OPEN 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.OBJ_OPEN (0 ms) 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.PUT_OBJECT 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.PUT_OBJECT (0 ms) 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.GET_OBJECT 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.GET_OBJECT (0 ms) 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.STAT_OBJECT 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.STAT_OBJECT (0 ms) 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.WRITE_READ_VERIFY 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.WRITE_READ_VERIFY (0 ms) 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.WRITEV 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.WRITEV (0 ms) 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READV 2024-05-09T20:19:34.297 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READV (0 ms) 2024-05-09T20:19:34.298 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.READV_AFTER_WRITEV 2024-05-09T20:19:34.298 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.READV_AFTER_WRITEV (0 ms) 2024-05-09T20:19:34.298 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_OBJECT 2024-05-09T20:19:34.305 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_OBJECT (9 ms) 2024-05-09T20:19:34.305 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.DELETE_BUCKET 2024-05-09T20:19:34.365 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.DELETE_BUCKET (60 ms) 2024-05-09T20:19:34.365 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:34.365 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:34.365 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:34.365 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:34.365 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:34.366 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:34.370+0000 7fe91b5f1a80 -1 shutting down 2024-05-09T20:19:34.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:34.374+0000 7fe7f6b46640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60221, si_uid : 1000, si_addr0x3e80000eb3d, si_status0 2024-05-09T20:19:34.529 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (164 ms) 2024-05-09T20:19:34.529 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 19 tests from LibRGW (789 ms total) 2024-05-09T20:19:34.529 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:34.529 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:34.529 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 19 tests from 1 test suite ran. (789 ms total) 2024-05-09T20:19:34.529 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 19 tests. 2024-05-09T20:19:34.540 INFO:tasks.workunit.client.0.smithi063.stdout:phase 6.1 2024-05-09T20:19:34.563 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] Running 10 tests from 1 test suite. 2024-05-09T20:19:34.564 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment set-up. 2024-05-09T20:19:34.564 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 10 tests from LibRGW 2024-05-09T20:19:34.564 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.INIT 2024-05-09T20:19:34.573 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:34.573 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:34.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:34.986+0000 7f4083079a80 -1 LDAP not started since no server URIs were provided in the configuration. 2024-05-09T20:19:35.114 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.INIT (551 ms) 2024-05-09T20:19:35.115 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.MOUNT 2024-05-09T20:19:35.116 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.MOUNT (2 ms) 2024-05-09T20:19:35.116 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CREATE_BUCKETS 2024-05-09T20:19:35.151 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CREATE_BUCKETS (35 ms) 2024-05-09T20:19:35.151 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.LOOKUP_BUCKETS 2024-05-09T20:19:35.162 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.LOOKUP_BUCKETS (11 ms) 2024-05-09T20:19:35.162 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.TOPDIR_RENAME 2024-05-09T20:19:35.194 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.TOPDIR_RENAME (32 ms) 2024-05-09T20:19:35.194 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SUBDIR_RENAME 2024-05-09T20:19:35.233 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SUBDIR_RENAME (39 ms) 2024-05-09T20:19:35.233 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CROSS_BUCKET_RENAME 2024-05-09T20:19:35.274 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CROSS_BUCKET_RENAME (41 ms) 2024-05-09T20:19:35.275 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.CLEANUP 2024-05-09T20:19:35.275 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.CLEANUP (0 ms) 2024-05-09T20:19:35.275 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.UMOUNT 2024-05-09T20:19:35.275 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.UMOUNT (0 ms) 2024-05-09T20:19:35.275 INFO:tasks.workunit.client.0.smithi063.stdout:[ RUN ] LibRGW.SHUTDOWN 2024-05-09T20:19:35.275 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.282+0000 7f4083079a80 -1 shutting down 2024-05-09T20:19:35.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.282+0000 7f3f5e5ce640 -1 received signal: User defined signal 1, si_code : -6, si_value (int): 0, si_value (ptr): 0, si_errno: 0, si_pid : 60865, si_uid : 1000, si_addr0x3e80000edc1, si_status0 2024-05-09T20:19:35.347 INFO:tasks.workunit.client.0.smithi063.stdout:[ OK ] LibRGW.SHUTDOWN (72 ms) 2024-05-09T20:19:35.347 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] 10 tests from LibRGW (783 ms total) 2024-05-09T20:19:35.347 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:35.347 INFO:tasks.workunit.client.0.smithi063.stdout:[----------] Global test environment tear-down 2024-05-09T20:19:35.347 INFO:tasks.workunit.client.0.smithi063.stdout:[==========] 10 tests from 1 test suite ran. (783 ms total) 2024-05-09T20:19:35.347 INFO:tasks.workunit.client.0.smithi063.stdout:[ PASSED ] 10 tests. 2024-05-09T20:19:35.354 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:19:35.355 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:19:35.366 INFO:tasks.workunit:Running workunits matching rgw/test_awssdkv4_sig.sh on client.0... 2024-05-09T20:19:35.367 INFO:tasks.workunit:Running workunit rgw/test_awssdkv4_sig.sh... 2024-05-09T20:19:35.368 DEBUG:teuthology.orchestra.run.smithi063:workunit test rgw/test_awssdkv4_sig.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_awssdkv4_sig.sh 2024-05-09T20:19:35.418 INFO:tasks.workunit.client.0.smithi063.stderr:+ [ -z ] 2024-05-09T20:19:35.418 INFO:tasks.workunit.client.0.smithi063.stderr:+ export AWS_ACCESS_KEY_ID=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-05-09T20:19:35.418 INFO:tasks.workunit.client.0.smithi063.stderr:+ export AWS_SECRET_ACCESS_KEY=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-05-09T20:19:35.418 INFO:tasks.workunit.client.0.smithi063.stderr:+ radosgw-admin user create --uid ceph-test-maven --access-key lNCnR47C2g+ZidCWBAUuwfSAA7Q= --secret tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== --display-name maven test user --email sigv4@example.com 2024-05-09T20:19:35.455 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:19:35.455 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:19:35.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.506+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 realm 2024-05-09T20:19:35.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:19:35.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:19:35.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:19:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.510+0000 7fd14b41c980 20 searching for the correct realm 2024-05-09T20:19:35.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.514+0000 7fd148f30640 0 -- 172.21.15.63:0/2777126264 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fd14003bc00 msgr2=0x7fd140032970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:19:35.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.730+0000 7fd148f30640 0 -- 172.21.15.63:0/2777126264 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fd14003bc00 msgr2=0x7fd140032970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:19:35.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:19:35.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:19:35.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:19:35.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:19:35.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:19:35.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:19:35.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:19:35.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.742+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:19:35.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 zone default found 2024-05-09T20:19:35.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 4 Realm: () 2024-05-09T20:19:35.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:19:35.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:19:35.738 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:19:35.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 zonegroup default 2024-05-09T20:19:35.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 started sync module instance, tier type = 2024-05-09T20:19:35.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.746+0000 7fd14b41c980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:19:35.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.770+0000 7fd14b41c980 20 add_watcher() i=0 2024-05-09T20:19:35.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.770+0000 7fd14b41c980 20 add_watcher() i=1 2024-05-09T20:19:35.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.770+0000 7fd14b41c980 20 add_watcher() i=2 2024-05-09T20:19:35.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.770+0000 7fd14b41c980 20 add_watcher() i=3 2024-05-09T20:19:35.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.770+0000 7fd14b41c980 20 add_watcher() i=4 2024-05-09T20:19:35.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.770+0000 7fd14b41c980 20 add_watcher() i=5 2024-05-09T20:19:35.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.770+0000 7fd14b41c980 20 add_watcher() i=6 2024-05-09T20:19:35.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.774+0000 7fd14b41c980 20 add_watcher() i=7 2024-05-09T20:19:35.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.774+0000 7fd14b41c980 2 all 8 watchers are set, enabling cache 2024-05-09T20:19:35.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.778+0000 7fcfff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:19:35.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.778+0000 7fd14b41c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:19:35.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.778+0000 7fd14b41c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:19:35.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.778+0000 7fd14b41c980 5 note: GC not initialized 2024-05-09T20:19:35.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.778+0000 7fcffe7e4640 20 reqs_thread_entry: start 2024-05-09T20:19:35.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.778+0000 7fd14b41c980 20 init_complete bucket index max shards: 11 2024-05-09T20:19:35.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.778+0000 7fd14b41c980 20 Filter name: none 2024-05-09T20:19:35.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fcff77fe640 20 reqs_thread_entry: start 2024-05-09T20:19:35.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache get: name=default.rgw.meta+users.uid+ceph-test-maven : miss 2024-05-09T20:19:35.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x0 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 adding default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache get: name=default.rgw.meta+users.email+sigv4@example.com : miss 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x0 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 adding default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : miss 2024-05-09T20:19:35.775 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 20 rados->read ofs=0 len=0 2024-05-09T20:19:35.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:19:35.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x0 2024-05-09T20:19:35.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 adding default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-05-09T20:19:35.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-05-09T20:19:35.776 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache get: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= : hit (negative entry) 2024-05-09T20:19:35.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-05-09T20:19:35.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-05-09T20:19:35.777 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.782+0000 7fd14b41c980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:ceph-test-maven, ofs0, ns] 2024-05-09T20:19:35.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.786+0000 7fd01ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791310 cookie 94529307490336 notifier 5465 bl.length()=677 2024-05-09T20:19:35.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.786+0000 7fd01ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ceph-test-maven info.flags=0x17 2024-05-09T20:19:35.778 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.786+0000 7fd01ffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ceph-test-maven to cache LRU end 2024-05-09T20:19:35.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.790+0000 7fd14b41c980 10 cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-05-09T20:19:35.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.790+0000 7fd14b41c980 10 moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-05-09T20:19:35.783 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.790+0000 7fd14b41c980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:sigv4@example.com, ofs0, ns] 2024-05-09T20:19:35.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.790+0000 7fd01ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791350 cookie 94529307476304 notifier 5465 bl.length()=187 2024-05-09T20:19:35.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.790+0000 7fd01ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sigv4@example.com info.flags=0x7 2024-05-09T20:19:35.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.790+0000 7fd01ffff640 10 rgw watcher librados: moving default.rgw.meta+users.email+sigv4@example.com to cache LRU end 2024-05-09T20:19:35.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.794+0000 7fd14b41c980 10 cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-05-09T20:19:35.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.794+0000 7fd14b41c980 10 moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-05-09T20:19:35.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.794+0000 7fd14b41c980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:lNCnR47C2g+ZidCWBAUuwfSAA7Q=, ofs0, ns] 2024-05-09T20:19:35.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.798+0000 7fd01ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1078036791351 cookie 94529307476304 notifier 5465 bl.length()=197 2024-05-09T20:19:35.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.798+0000 7fd01ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= info.flags=0x7 2024-05-09T20:19:35.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.798+0000 7fd01ffff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+lNCnR47C2g+ZidCWBAUuwfSAA7Q= to cache LRU end 2024-05-09T20:19:35.792 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:19:35.792 INFO:tasks.workunit.client.0.smithi063.stdout: "user_id": "ceph-test-maven", 2024-05-09T20:19:35.792 INFO:tasks.workunit.client.0.smithi063.stdout: "display_name": "maven test user", 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "email": "sigv4@example.com", 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "suspended": 0, 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "subusers": [], 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "keys": [ 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: { 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "user": "ceph-test-maven", 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "access_key": "lNCnR47C2g+ZidCWBAUuwfSAA7Q=", 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "secret_key": "tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g==", 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "active": true, 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "create_date": "2024-05-09T20:19:35.787739Z" 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: ], 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "swift_keys": [], 2024-05-09T20:19:35.793 INFO:tasks.workunit.client.0.smithi063.stdout: "caps": [], 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "default_placement": "", 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "placement_tags": [], 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_quota": { 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "enabled": false, 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size": -1, 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "max_objects": -1 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "user_quota": { 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "enabled": false, 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:19:35.794 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size": -1, 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "max_objects": -1 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "type": "rgw", 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "account_id": "", 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "path": "/", 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "create_date": "2024-05-09T20:19:35.787691Z", 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "tags": [], 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: "group_ids": [] 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:19:35.795 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:19:35.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.798+0000 7fcfff7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:19:35.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.806+0000 7fd14b41c980 20 remove_watcher() i=0 2024-05-09T20:19:35.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.806+0000 7fd14b41c980 2 removed watcher, disabling cache 2024-05-09T20:19:35.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.806+0000 7fd14b41c980 20 remove_watcher() i=1 2024-05-09T20:19:35.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.810+0000 7fd14b41c980 20 remove_watcher() i=2 2024-05-09T20:19:35.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.810+0000 7fd14b41c980 20 remove_watcher() i=3 2024-05-09T20:19:35.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.814+0000 7fd14b41c980 20 remove_watcher() i=4 2024-05-09T20:19:35.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.814+0000 7fd14b41c980 20 remove_watcher() i=5 2024-05-09T20:19:35.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.818+0000 7fd14b41c980 20 remove_watcher() i=6 2024-05-09T20:19:35.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:19:35.818+0000 7fd14b41c980 20 remove_watcher() i=7 2024-05-09T20:19:35.827 INFO:tasks.workunit.client.0.smithi063.stderr:+ [ -z ] 2024-05-09T20:19:35.827 INFO:tasks.workunit.client.0.smithi063.stderr:+ [ -z /home/ubuntu/cephtest ] 2024-05-09T20:19:35.827 INFO:tasks.workunit.client.0.smithi063.stderr:+ cat /home/ubuntu/cephtest/url_file 2024-05-09T20:19:35.828 INFO:tasks.workunit.client.0.smithi063.stderr:+ export RGW_HTTP_ENDPOINT_URL=http://smithi063.front.sepia.ceph.com:80 2024-05-09T20:19:35.828 INFO:tasks.workunit.client.0.smithi063.stderr:+ [ -z /home/ubuntu/cephtest/clone.client.0 ] 2024-05-09T20:19:35.828 INFO:tasks.workunit.client.0.smithi063.stderr:+ cd /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum 2024-05-09T20:19:35.828 INFO:tasks.workunit.client.0.smithi063.stderr:+ ./mvnw clean package 2024-05-09T20:19:35.835 INFO:tasks.workunit.client.0.smithi063.stdout:Warning: JAVA_HOME environment variable is not set. 2024-05-09T20:19:40.187 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Scanning for projects... 2024-05-09T20:19:40.325 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom 2024-05-09T20:19:40.426 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/5.6 kB Progress (1): 2.8/5.6 kB Progress (1): 4.1/5.6 kB Progress (1): 5.5/5.6 kB Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.1/junit-bom-5.10.1.pom (5.6 kB at 54 kB/s) 2024-05-09T20:19:40.450 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom 2024-05-09T20:19:40.491 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/83 kB Progress (1): 33/83 kB Progress (1): 49/83 kB Progress (1): 66/83 kB Progress (1): 82/83 kB Progress (1): 83 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom/2.20.43/bom-2.20.43.pom (83 kB at 2.0 MB/s) 2024-05-09T20:19:40.504 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom 2024-05-09T20:19:40.528 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/77 kB Progress (1): 33/77 kB Progress (1): 49/77 kB Progress (1): 66/77 kB Progress (1): 77 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-sdk-java-pom/2.20.43/aws-sdk-java-pom-2.20.43.pom (77 kB at 3.2 MB/s) 2024-05-09T20:19:40.566 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:40.566 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-05-09T20:19:40.566 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Building jcksum 1.0 2024-05-09T20:19:40.567 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] from pom.xml 2024-05-09T20:19:40.567 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-05-09T20:19:40.571 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom 2024-05-09T20:19:40.591 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.pom (5.3 kB at 265 kB/s) 2024-05-09T20:19:40.601 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom 2024-05-09T20:19:40.620 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/35/maven-plugins-35.pom (9.9 kB at 495 kB/s) 2024-05-09T20:19:40.625 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom 2024-05-09T20:19:40.646 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/35/maven-parent-35.pom (45 kB at 2.3 MB/s) 2024-05-09T20:19:40.656 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom 2024-05-09T20:19:40.676 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/25/apache-25.pom (21 kB at 1.1 MB/s) 2024-05-09T20:19:40.693 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar 2024-05-09T20:19:40.714 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/3.2.0/maven-clean-plugin-3.2.0.jar (36 kB at 1.7 MB/s) 2024-05-09T20:19:40.724 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom 2024-05-09T20:19:40.744 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.pom (8.2 kB at 408 kB/s) 2024-05-09T20:19:40.748 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom 2024-05-09T20:19:40.768 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/39/maven-plugins-39.pom (8.1 kB at 405 kB/s) 2024-05-09T20:19:40.771 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom 2024-05-09T20:19:40.792 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/48 kB Progress (1): 33/48 kB Progress (1): 48 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/39/maven-parent-39.pom (48 kB at 2.3 MB/s) 2024-05-09T20:19:40.797 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom 2024-05-09T20:19:40.817 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom (21 kB at 1.1 MB/s) 2024-05-09T20:19:40.825 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar 2024-05-09T20:19:40.846 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/3.3.1/maven-resources-plugin-3.3.1.jar (31 kB at 1.5 MB/s) 2024-05-09T20:19:40.857 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom 2024-05-09T20:19:40.877 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 9.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.pom (9.8 kB at 491 kB/s) 2024-05-09T20:19:40.887 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar 2024-05-09T20:19:40.909 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/66 kB Progress (1): 33/66 kB Progress (1): 49/66 kB Progress (1): 66/66 kB Progress (1): 66 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.11.0/maven-compiler-plugin-3.11.0.jar (66 kB at 3.0 MB/s) 2024-05-09T20:19:40.920 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom 2024-05-09T20:19:40.940 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.pom (5.3 kB at 266 kB/s) 2024-05-09T20:19:40.944 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom 2024-05-09T20:19:40.963 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/3.2.3/surefire-3.2.3.pom (22 kB at 1.2 MB/s) 2024-05-09T20:19:40.969 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom 2024-05-09T20:19:40.989 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/50 kB Progress (1): 33/50 kB Progress (1): 49/50 kB Progress (1): 50 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/41/maven-parent-41.pom (50 kB at 2.5 MB/s) 2024-05-09T20:19:40.996 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom 2024-05-09T20:19:41.015 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/31/apache-31.pom (24 kB at 1.2 MB/s) 2024-05-09T20:19:41.024 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom 2024-05-09T20:19:41.043 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.3/junit-bom-5.9.3.pom (5.6 kB at 296 kB/s) 2024-05-09T20:19:41.050 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar 2024-05-09T20:19:41.070 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/45 kB Progress (1): 32/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/3.2.3/maven-surefire-plugin-3.2.3.jar (45 kB at 2.1 MB/s) 2024-05-09T20:19:41.077 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom 2024-05-09T20:19:41.098 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.pom (6.8 kB at 322 kB/s) 2024-05-09T20:19:41.101 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom 2024-05-09T20:19:41.121 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 9.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/37/maven-plugins-37.pom (9.9 kB at 521 kB/s) 2024-05-09T20:19:41.124 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom 2024-05-09T20:19:41.154 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/46 kB Progress (1): 33/46 kB Progress (1): 46 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/37/maven-parent-37.pom (46 kB at 1.5 MB/s) 2024-05-09T20:19:41.160 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom 2024-05-09T20:19:41.208 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/27/apache-27.pom (20 kB at 424 kB/s) 2024-05-09T20:19:41.224 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar 2024-05-09T20:19:41.261 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/3.3.0/maven-jar-plugin-3.3.0.jar (27 kB at 739 kB/s) 2024-05-09T20:19:41.304 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom 2024-05-09T20:19:41.327 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.pom (6.1 kB at 265 kB/s) 2024-05-09T20:19:41.333 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom 2024-05-09T20:19:41.356 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/services/2.20.43/services-2.20.43.pom (21 kB at 888 kB/s) 2024-05-09T20:19:41.374 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom 2024-05-09T20:19:41.398 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/bom-internal/2.20.43/bom-internal-2.20.43.pom (20 kB at 800 kB/s) 2024-05-09T20:19:41.423 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom 2024-05-09T20:19:41.445 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.pom (4.0 kB at 182 kB/s) 2024-05-09T20:19:41.450 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom 2024-05-09T20:19:41.473 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocols/2.20.43/protocols-2.20.43.pom (1.6 kB at 68 kB/s) 2024-05-09T20:19:41.476 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom 2024-05-09T20:19:41.496 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/core/2.20.43/core-2.20.43.pom (2.2 kB at 117 kB/s) 2024-05-09T20:19:41.510 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom 2024-05-09T20:19:41.532 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.pom (3.6 kB at 165 kB/s) 2024-05-09T20:19:41.545 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom 2024-05-09T20:19:41.576 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.pom (3.2 kB at 104 kB/s) 2024-05-09T20:19:41.584 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom 2024-05-09T20:19:41.617 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.pom (11 kB at 332 kB/s) 2024-05-09T20:19:41.624 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom 2024-05-09T20:19:41.661 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.pom (1.7 kB at 47 kB/s) 2024-05-09T20:19:41.664 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom 2024-05-09T20:19:41.701 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.pom (3.8 kB at 103 kB/s) 2024-05-09T20:19:41.710 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom 2024-05-09T20:19:41.751 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.pom (4.9 kB at 120 kB/s) 2024-05-09T20:19:41.758 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom 2024-05-09T20:19:41.800 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.pom (1.2 kB at 28 kB/s) 2024-05-09T20:19:41.806 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom 2024-05-09T20:19:41.847 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom (3.8 kB at 94 kB/s) 2024-05-09T20:19:41.852 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom 2024-05-09T20:19:41.885 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.30/slf4j-parent-1.7.30.pom (14 kB at 406 kB/s) 2024-05-09T20:19:41.893 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom 2024-05-09T20:19:41.919 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.pom (3.7 kB at 136 kB/s) 2024-05-09T20:19:41.929 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom 2024-05-09T20:19:41.949 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.pom (2.5 kB at 117 kB/s) 2024-05-09T20:19:41.958 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom 2024-05-09T20:19:41.979 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.pom (2.9 kB at 139 kB/s) 2024-05-09T20:19:41.989 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom 2024-05-09T20:19:42.010 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 7.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.pom (7.4 kB at 353 kB/s) 2024-05-09T20:19:42.019 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom 2024-05-09T20:19:42.041 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.pom (5.5 kB at 248 kB/s) 2024-05-09T20:19:42.051 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom 2024-05-09T20:19:42.073 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.pom (3.5 kB at 159 kB/s) 2024-05-09T20:19:42.084 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom 2024-05-09T20:19:42.112 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.pom (4.4 kB at 157 kB/s) 2024-05-09T20:19:42.117 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom 2024-05-09T20:19:42.138 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party/2.20.43/third-party-2.20.43.pom (2.4 kB at 115 kB/s) 2024-05-09T20:19:42.146 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom 2024-05-09T20:19:42.167 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.pom (6.4 kB at 322 kB/s) 2024-05-09T20:19:42.177 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom 2024-05-09T20:19:42.198 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.pom (6.1 kB at 289 kB/s) 2024-05-09T20:19:42.207 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom 2024-05-09T20:19:42.230 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.pom (3.0 kB at 129 kB/s) 2024-05-09T20:19:42.239 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom 2024-05-09T20:19:42.260 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.pom (3.5 kB at 165 kB/s) 2024-05-09T20:19:42.271 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom 2024-05-09T20:19:42.291 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.pom (4.1 kB at 186 kB/s) 2024-05-09T20:19:42.296 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom 2024-05-09T20:19:42.317 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-clients/2.20.43/http-clients-2.20.43.pom (1.8 kB at 85 kB/s) 2024-05-09T20:19:42.327 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom 2024-05-09T20:19:42.352 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.pom (6.6 kB at 264 kB/s) 2024-05-09T20:19:42.357 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom 2024-05-09T20:19:42.382 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/16 kB Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.13/httpcomponents-client-4.5.13.pom (16 kB at 655 kB/s) 2024-05-09T20:19:42.388 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom 2024-05-09T20:19:42.417 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/35 kB Progress (1): 33/35 kB Progress (1): 35 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/11/httpcomponents-parent-11.pom (35 kB at 1.2 MB/s) 2024-05-09T20:19:42.425 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom 2024-05-09T20:19:42.452 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/21/apache-21.pom (17 kB at 634 kB/s) 2024-05-09T20:19:42.461 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom 2024-05-09T20:19:42.489 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.pom (5.0 kB at 171 kB/s) 2024-05-09T20:19:42.494 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom 2024-05-09T20:19:42.525 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.13/httpcomponents-core-4.4.13.pom (13 kB at 411 kB/s) 2024-05-09T20:19:42.535 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom 2024-05-09T20:19:42.567 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.11/commons-codec-1.11.pom (14 kB at 436 kB/s) 2024-05-09T20:19:42.572 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom 2024-05-09T20:19:42.603 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/68 kB Progress (1): 33/68 kB Progress (1): 49/68 kB Progress (1): 66/68 kB Progress (1): 68 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/42/commons-parent-42.pom (68 kB at 2.2 MB/s) 2024-05-09T20:19:42.624 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2024-05-09T20:19:42.646 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom (16 kB at 712 kB/s) 2024-05-09T20:19:42.657 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom 2024-05-09T20:19:42.681 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.pom (15 kB at 643 kB/s) 2024-05-09T20:19:42.686 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom 2024-05-09T20:19:42.720 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/79 kB Progress (1): 32/79 kB Progress (1): 49/79 kB Progress (1): 65/79 kB Progress (1): 79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/52/commons-parent-52.pom (79 kB at 2.3 MB/s) 2024-05-09T20:19:42.730 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom 2024-05-09T20:19:42.760 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 614 kB/s) 2024-05-09T20:19:42.769 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom 2024-05-09T20:19:42.799 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.pom (20 kB at 644 kB/s) 2024-05-09T20:19:42.805 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom 2024-05-09T20:19:42.848 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 66/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/65/commons-parent-65.pom (78 kB at 1.8 MB/s) 2024-05-09T20:19:42.862 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom 2024-05-09T20:19:42.898 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.pom (3.8 kB at 107 kB/s) 2024-05-09T20:19:42.903 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom 2024-05-09T20:19:42.946 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.28/slf4j-parent-1.7.28.pom (14 kB at 314 kB/s) 2024-05-09T20:19:42.953 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom 2024-05-09T20:19:42.998 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.pom (3.2 kB at 71 kB/s) 2024-05-09T20:19:43.003 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom 2024-05-09T20:19:43.052 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.pom (3.2 kB at 65 kB/s) 2024-05-09T20:19:43.057 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom 2024-05-09T20:19:43.097 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.pom (2.0 kB at 52 kB/s) 2024-05-09T20:19:43.102 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom 2024-05-09T20:19:43.139 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.pom (2.8 kB at 77 kB/s) 2024-05-09T20:19:43.144 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom 2024-05-09T20:19:43.181 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.pom (1.5 kB at 41 kB/s) 2024-05-09T20:19:43.186 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom 2024-05-09T20:19:43.228 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.pom (3.0 kB at 72 kB/s) 2024-05-09T20:19:43.234 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom 2024-05-09T20:19:43.283 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.pom (3.2 kB at 65 kB/s) 2024-05-09T20:19:43.289 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom 2024-05-09T20:19:43.336 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.pom (3.2 kB at 68 kB/s) 2024-05-09T20:19:43.343 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom 2024-05-09T20:19:43.383 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.pom (1.0 kB at 26 kB/s) 2024-05-09T20:19:43.392 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom 2024-05-09T20:19:43.433 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 964 B Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.pom (964 B at 24 kB/s) 2024-05-09T20:19:43.441 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom 2024-05-09T20:19:43.468 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.pom (3.1 kB at 109 kB/s) 2024-05-09T20:19:43.474 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom 2024-05-09T20:19:43.505 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.pom (3.0 kB at 98 kB/s) 2024-05-09T20:19:43.511 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom 2024-05-09T20:19:43.535 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.pom (3.4 kB at 143 kB/s) 2024-05-09T20:19:43.541 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom 2024-05-09T20:19:43.565 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.pom (3.4 kB at 137 kB/s) 2024-05-09T20:19:43.570 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom 2024-05-09T20:19:43.589 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.pom (3.0 kB at 159 kB/s) 2024-05-09T20:19:43.638 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar 2024-05-09T20:19:44.468 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.1/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.2/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.3/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.4/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.5/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.6/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.7/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.8/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 0.9/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.0/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.1/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.2/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.3/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.4/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.5/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.6/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.7/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.8/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 1.9/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.0/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.1/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.2/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.3/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.4/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.5/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.6/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.7/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.8/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 2.9/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.0/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.1/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.2/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3/3.3 MB Progress (1): 3.3 MB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/s3/2.20.43/s3-2.20.43.jar (3.3 MB at 4.0 MB/s) 2024-05-09T20:19:44.471 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar 2024-05-09T20:19:44.472 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar 2024-05-09T20:19:44.472 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar 2024-05-09T20:19:44.473 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar 2024-05-09T20:19:44.474 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar 2024-05-09T20:19:44.500 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/71 kB Progress (1): 33/71 kB Progress (1): 49/71 kB Progress (1): 63/71 kB Progress (1): 71 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-query-protocol/2.20.43/aws-query-protocol-2.20.43.jar (71 kB at 2.4 MB/s) 2024-05-09T20:19:44.500 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar 2024-05-09T20:19:44.519 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/crt-core/2.20.43/crt-core-2.20.43.jar (16 kB at 356 kB/s) 2024-05-09T20:19:44.520 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar 2024-05-09T20:19:44.538 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/35 kB Progress (1): 2.8/35 kB Progress (1): 4.1/35 kB Progress (1): 5.5/35 kB Progress (1): 6.9/35 kB Progress (1): 8.3/35 kB Progress (1): 9.7/35 kB Progress (1): 11/35 kB Progress (1): 12/35 kB Progress (1): 14/35 kB Progress (1): 15/35 kB Progress (1): 16/35 kB Progress (1): 17/35 kB Progress (1): 19/35 kB Progress (1): 20/35 kB Progress (1): 21/35 kB Progress (1): 23/35 kB Progress (2): 23/35 kB | 16/98 kB Progress (2): 24/35 kB | 16/98 kB Progress (2): 25/35 kB | 16/98 kB Progress (2): 27/35 kB | 16/98 kB Progress (2): 28/35 kB | 16/98 kB Progress (2): 30/35 kB | 16/98 kB Progress (2): 31/35 kB | 16/98 kB Progress (2): 32/35 kB | 16/98 kB Progress (2): 34/35 kB | 16/98 kB Progress (2): 35/35 kB | 16/98 kB Progress (2): 35 kB | 16/98 kB Progress (3): 35 kB | 16/98 kB | 16/782 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/protocol-core/2.20.43/protocol-core-2.20.43.jar (35 kB at 525 kB/s) 2024-05-09T20:19:44.539 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar 2024-05-09T20:19:44.550 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 16/98 kB | 32/782 kB Progress (2): 33/98 kB | 32/782 kB Progress (2): 33/98 kB | 49/782 kB Progress (2): 33/98 kB | 65/782 kB Progress (2): 33/98 kB | 81/782 kB Progress (2): 33/98 kB | 98/782 kB Progress (3): 33/98 kB | 98/782 kB | 1.4/49 kB Progress (3): 33/98 kB | 98/782 kB | 2.8/49 kB Progress (4): 33/98 kB | 98/782 kB | 2.8/49 kB | 1.4/16 kB Progress (4): 33/98 kB | 98/782 kB | 4.1/49 kB | 1.4/16 kB Progress (4): 33/98 kB | 98/782 kB | 4.1/49 kB | 2.8/16 kB Progress (4): 33/98 kB | 114/782 kB | 4.1/49 kB | 2.8/16 kB Progress (4): 33/98 kB | 114/782 kB | 5.5/49 kB | 2.8/16 kB Progress (4): 33/98 kB | 114/782 kB | 5.5/49 kB | 4.1/16 kB Progress (4): 33/98 kB | 114/782 kB | 6.9/49 kB | 4.1/16 kB Progress (4): 33/98 kB | 114/782 kB | 6.9/49 kB | 5.5/16 kB Progress (4): 33/98 kB | 114/782 kB | 8.3/49 kB | 5.5/16 kB Progress (4): 33/98 kB | 114/782 kB | 8.3/49 kB | 6.9/16 kB Progress (4): 33/98 kB | 114/782 kB | 9.7/49 kB | 6.9/16 kB Progress (4): 33/98 kB | 114/782 kB | 9.7/49 kB | 8.3/16 kB Progress (4): 33/98 kB | 114/782 kB | 11/49 kB | 8.3/16 kB Progress (4): 33/98 kB | 114/782 kB | 11/49 kB | 9.7/16 kB Progress (4): 33/98 kB | 114/782 kB | 12/49 kB | 9.7/16 kB Progress (4): 33/98 kB | 130/782 kB | 12/49 kB | 9.7/16 kB Progress (4): 33/98 kB | 130/782 kB | 14/49 kB | 9.7/16 kB Progress (4): 33/98 kB | 130/782 kB | 14/49 kB | 11/16 kB Progress (4): 33/98 kB | 130/782 kB | 15/49 kB | 11/16 kB Progress (4): 33/98 kB | 130/782 kB | 15/49 kB | 12/16 kB Progress (4): 33/98 kB | 130/782 kB | 16/49 kB | 12/16 kB Progress (4): 33/98 kB | 130/782 kB | 16/49 kB | 14/16 kB Progress (4): 33/98 kB | 130/782 kB | 17/49 kB | 14/16 kB Progress (4): 33/98 kB | 130/782 kB | 17/49 kB | 15/16 kB Progress (4): 33/98 kB | 130/782 kB | 19/49 kB | 15/16 kB Progress (4): 33/98 kB | 147/782 kB | 19/49 kB | 15/16 kB Progress (4): 33/98 kB | 147/782 kB | 19/49 kB | 16 kB Progress (4): 33/98 kB | 147/782 kB | 20/49 kB | 16 kB Progress (4): 49/98 kB | 147/782 kB | 20/49 kB | 16 kB Progress (4): 49/98 kB | 147/782 kB | 21/49 kB | 16 kB Progress (4): 49/98 kB | 147/782 kB | 23/49 kB | 16 kB Progress (4): 49/98 kB | 147/782 kB | 24/49 kB | 16 kB Progress (4): 49/98 kB | 147/782 kB | 25/49 kB | 16 kB Progress (4): 49/98 kB | 147/782 kB | 27/49 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/arns/2.20.43/arns-2.20.43.jar (16 kB at 202 kB/s) 2024-05-09T20:19:44.551 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 49/98 kB | 147/782 kB | 28/49 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar 2024-05-09T20:19:44.570 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 49/98 kB | 147/782 kB | 30/49 kB Progress (3): 49/98 kB | 147/782 kB | 31/49 kB Progress (3): 49/98 kB | 147/782 kB | 32/49 kB Progress (3): 49/98 kB | 147/782 kB | 34/49 kB Progress (3): 49/98 kB | 147/782 kB | 35/49 kB Progress (3): 49/98 kB | 147/782 kB | 36/49 kB Progress (3): 49/98 kB | 147/782 kB | 38/49 kB Progress (3): 49/98 kB | 163/782 kB | 38/49 kB Progress (3): 49/98 kB | 163/782 kB | 39/49 kB Progress (3): 66/98 kB | 163/782 kB | 39/49 kB Progress (3): 66/98 kB | 163/782 kB | 41/49 kB Progress (4): 66/98 kB | 163/782 kB | 41/49 kB | 1.4/11 kB Progress (4): 66/98 kB | 163/782 kB | 42/49 kB | 1.4/11 kB Progress (4): 66/98 kB | 163/782 kB | 42/49 kB | 2.8/11 kB Progress (4): 66/98 kB | 180/782 kB | 42/49 kB | 2.8/11 kB Progress (4): 82/98 kB | 180/782 kB | 42/49 kB | 2.8/11 kB Progress (4): 82/98 kB | 180/782 kB | 42/49 kB | 4.1/11 kB Progress (4): 82/98 kB | 180/782 kB | 43/49 kB | 4.1/11 kB Progress (4): 82/98 kB | 180/782 kB | 43/49 kB | 5.5/11 kB Progress (4): 82/98 kB | 180/782 kB | 45/49 kB | 5.5/11 kB Progress (4): 98 kB | 180/782 kB | 45/49 kB | 5.5/11 kB Progress (4): 98 kB | 196/782 kB | 45/49 kB | 5.5/11 kB Progress (4): 98 kB | 196/782 kB | 45/49 kB | 6.9/11 kB Progress (4): 98 kB | 196/782 kB | 46/49 kB | 6.9/11 kB Progress (4): 98 kB | 196/782 kB | 46/49 kB | 8.3/11 kB Progress (4): 98 kB | 196/782 kB | 48/49 kB | 8.3/11 kB Progress (4): 98 kB | 212/782 kB | 48/49 kB | 8.3/11 kB Progress (4): 98 kB | 212/782 kB | 48/49 kB | 9.7/11 kB Progress (4): 98 kB | 212/782 kB | 49/49 kB | 9.7/11 kB Progress (4): 98 kB | 212/782 kB | 49/49 kB | 11/11 kB Progress (4): 98 kB | 212/782 kB | 49 kB | 11/11 kB Progress (4): 98 kB | 229/782 kB | 49 kB | 11/11 kB Progress (4): 98 kB | 229/782 kB | 49 kB | 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-xml-protocol/2.20.43/aws-xml-protocol-2.20.43.jar (98 kB at 971 kB/s) 2024-05-09T20:19:44.570 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 245/782 kB | 49 kB | 11 kB Progress (4): 245/782 kB | 49 kB | 11 kB | 16/205 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/profiles/2.20.43/profiles-2.20.43.jar (49 kB at 513 kB/s) 2024-05-09T20:19:44.570 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/reactivestreams/reactive-streams/1.0.3/reactive-streams-1.0.3.jar (11 kB at 121 kB/s) 2024-05-09T20:19:44.570 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar 2024-05-09T20:19:44.570 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar 2024-05-09T20:19:44.570 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar 2024-05-09T20:19:44.591 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 262/782 kB | 16/205 kB Progress (2): 278/782 kB | 16/205 kB Progress (2): 294/782 kB | 16/205 kB Progress (2): 311/782 kB | 16/205 kB Progress (2): 311/782 kB | 33/205 kB Progress (2): 327/782 kB | 33/205 kB Progress (2): 343/782 kB | 33/205 kB Progress (2): 343/782 kB | 49/205 kB Progress (2): 360/782 kB | 49/205 kB Progress (2): 360/782 kB | 66/205 kB Progress (2): 376/782 kB | 66/205 kB Progress (2): 393/782 kB | 66/205 kB Progress (2): 393/782 kB | 82/205 kB Progress (2): 402/782 kB | 82/205 kB Progress (2): 402/782 kB | 98/205 kB Progress (3): 402/782 kB | 98/205 kB | 16/30 kB Progress (3): 418/782 kB | 98/205 kB | 16/30 kB Progress (4): 418/782 kB | 98/205 kB | 16/30 kB | 16/81 kB Progress (4): 418/782 kB | 98/205 kB | 16/30 kB | 32/81 kB Progress (4): 435/782 kB | 98/205 kB | 16/30 kB | 32/81 kB Progress (4): 435/782 kB | 98/205 kB | 30 kB | 32/81 kB Progress (5): 435/782 kB | 98/205 kB | 30 kB | 32/81 kB | 16/798 kB Progress (5): 435/782 kB | 98/205 kB | 30 kB | 49/81 kB | 16/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/eventstream/eventstream/1.0.1/eventstream-1.0.1.jar (30 kB at 260 kB/s) 2024-05-09T20:19:44.591 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar 2024-05-09T20:19:44.593 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (4): 435/782 kB | 98/205 kB | 65/81 kB | 16/798 kB Progress (4): 435/782 kB | 98/205 kB | 65/81 kB | 32/798 kB Progress (4): 451/782 kB | 98/205 kB | 65/81 kB | 32/798 kB Progress (4): 451/782 kB | 98/205 kB | 81 kB | 32/798 kB Progress (4): 451/782 kB | 98/205 kB | 81 kB | 49/798 kB Progress (4): 468/782 kB | 98/205 kB | 81 kB | 49/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/http-client-spi/2.20.43/http-client-spi-2.20.43.jar (81 kB at 689 kB/s) 2024-05-09T20:19:44.593 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar 2024-05-09T20:19:44.604 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 484/782 kB | 98/205 kB | 49/798 kB Progress (3): 500/782 kB | 98/205 kB | 49/798 kB Progress (3): 500/782 kB | 115/205 kB | 49/798 kB Progress (3): 500/782 kB | 131/205 kB | 49/798 kB Progress (3): 517/782 kB | 131/205 kB | 49/798 kB Progress (3): 517/782 kB | 147/205 kB | 49/798 kB Progress (3): 517/782 kB | 164/205 kB | 49/798 kB Progress (3): 533/782 kB | 164/205 kB | 49/798 kB Progress (3): 549/782 kB | 164/205 kB | 49/798 kB Progress (3): 549/782 kB | 180/205 kB | 49/798 kB Progress (3): 566/782 kB | 180/205 kB | 49/798 kB Progress (3): 566/782 kB | 197/205 kB | 49/798 kB Progress (3): 566/782 kB | 205 kB | 49/798 kB Progress (3): 566/782 kB | 205 kB | 65/798 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/auth/2.20.43/auth-2.20.43.jar (205 kB at 1.6 MB/s) 2024-05-09T20:19:44.604 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar 2024-05-09T20:19:44.610 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 582/782 kB | 65/798 kB Progress (2): 582/782 kB | 81/798 kB Progress (2): 582/782 kB | 98/798 kB Progress (2): 599/782 kB | 98/798 kB Progress (2): 615/782 kB | 98/798 kB Progress (2): 615/782 kB | 114/798 kB Progress (2): 631/782 kB | 114/798 kB Progress (2): 631/782 kB | 130/798 kB Progress (3): 631/782 kB | 130/798 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/annotations/2.20.43/annotations-2.20.43.jar (13 kB at 98 kB/s) 2024-05-09T20:19:44.610 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 631/782 kB | 147/798 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar 2024-05-09T20:19:44.624 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 631/782 kB | 147/798 kB | 16/182 kB Progress (3): 631/782 kB | 163/798 kB | 16/182 kB Progress (3): 648/782 kB | 163/798 kB | 16/182 kB Progress (3): 648/782 kB | 163/798 kB | 33/182 kB Progress (3): 648/782 kB | 180/798 kB | 33/182 kB Progress (3): 648/782 kB | 180/798 kB | 49/182 kB Progress (3): 648/782 kB | 196/798 kB | 49/182 kB Progress (3): 664/782 kB | 196/798 kB | 49/182 kB Progress (3): 664/782 kB | 212/798 kB | 49/182 kB Progress (3): 664/782 kB | 212/798 kB | 66/182 kB Progress (3): 664/782 kB | 212/798 kB | 82/182 kB Progress (3): 664/782 kB | 212/798 kB | 98/182 kB Progress (3): 681/782 kB | 212/798 kB | 98/182 kB Progress (3): 681/782 kB | 212/798 kB | 115/182 kB Progress (3): 697/782 kB | 212/798 kB | 115/182 kB Progress (3): 697/782 kB | 212/798 kB | 131/182 kB Progress (3): 697/782 kB | 212/798 kB | 147/182 kB Progress (3): 697/782 kB | 212/798 kB | 164/182 kB Progress (3): 713/782 kB | 212/798 kB | 164/182 kB Progress (3): 730/782 kB | 212/798 kB | 164/182 kB Progress (3): 730/782 kB | 229/798 kB | 164/182 kB Progress (3): 730/782 kB | 245/798 kB | 164/182 kB Progress (3): 746/782 kB | 245/798 kB | 164/182 kB Progress (3): 746/782 kB | 262/798 kB | 164/182 kB Progress (3): 746/782 kB | 278/798 kB | 164/182 kB Progress (3): 762/782 kB | 278/798 kB | 164/182 kB Progress (3): 762/782 kB | 294/798 kB | 164/182 kB Progress (4): 762/782 kB | 294/798 kB | 164/182 kB | 16/151 kB Progress (4): 762/782 kB | 311/798 kB | 164/182 kB | 16/151 kB Progress (4): 762/782 kB | 311/798 kB | 164/182 kB | 33/151 kB Progress (4): 762/782 kB | 327/798 kB | 164/182 kB | 33/151 kB Progress (4): 762/782 kB | 327/798 kB | 164/182 kB | 49/151 kB Progress (4): 779/782 kB | 327/798 kB | 164/182 kB | 49/151 kB Progress (4): 782 kB | 327/798 kB | 164/182 kB | 49/151 kB Progress (4): 782 kB | 343/798 kB | 164/182 kB | 49/151 kB Progress (4): 782 kB | 343/798 kB | 164/182 kB | 66/151 kB Progress (4): 782 kB | 360/798 kB | 164/182 kB | 66/151 kB Progress (4): 782 kB | 360/798 kB | 164/182 kB | 82/151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/sdk-core/2.20.43/sdk-core-2.20.43.jar (782 kB at 5.3 MB/s) 2024-05-09T20:19:44.625 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar 2024-05-09T20:19:44.627 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 360/798 kB | 164/182 kB | 98/151 kB Progress (3): 360/798 kB | 164/182 kB | 115/151 kB Progress (3): 376/798 kB | 164/182 kB | 115/151 kB Progress (3): 376/798 kB | 164/182 kB | 131/151 kB Progress (3): 393/798 kB | 164/182 kB | 131/151 kB Progress (3): 393/798 kB | 164/182 kB | 147/151 kB Progress (3): 402/798 kB | 164/182 kB | 147/151 kB Progress (3): 402/798 kB | 164/182 kB | 151 kB Progress (3): 418/798 kB | 164/182 kB | 151 kB Progress (3): 418/798 kB | 180/182 kB | 151 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/aws-core/2.20.43/aws-core-2.20.43.jar (151 kB at 998 kB/s) 2024-05-09T20:19:44.627 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 418/798 kB | 182 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar 2024-05-09T20:19:44.627 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 435/798 kB | 182 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/utils/2.20.43/utils-2.20.43.jar (182 kB at 1.2 MB/s) 2024-05-09T20:19:44.627 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 451/798 kB Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar 2024-05-09T20:19:44.632 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 468/798 kB Progress (1): 484/798 kB Progress (1): 500/798 kB Progress (1): 517/798 kB Progress (2): 517/798 kB | 2.7/27 kB Progress (2): 517/798 kB | 5.5/27 kB Progress (2): 517/798 kB | 6.9/27 kB Progress (2): 517/798 kB | 8.2/27 kB Progress (2): 517/798 kB | 9.6/27 kB Progress (2): 517/798 kB | 11/27 kB Progress (2): 517/798 kB | 12/27 kB Progress (2): 517/798 kB | 14/27 kB Progress (2): 517/798 kB | 15/27 kB Progress (2): 517/798 kB | 16/27 kB Progress (2): 517/798 kB | 18/27 kB Progress (2): 517/798 kB | 21/27 kB Progress (2): 517/798 kB | 25/27 kB Progress (2): 517/798 kB | 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/metrics-spi/2.20.43/metrics-spi-2.20.43.jar (27 kB at 174 kB/s) 2024-05-09T20:19:44.632 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar 2024-05-09T20:19:44.642 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 533/798 kB Progress (1): 549/798 kB Progress (1): 566/798 kB Progress (1): 582/798 kB Progress (1): 599/798 kB Progress (1): 615/798 kB Progress (1): 631/798 kB Progress (1): 648/798 kB Progress (1): 664/798 kB Progress (1): 681/798 kB Progress (1): 697/798 kB Progress (1): 713/798 kB Progress (1): 730/798 kB Progress (1): 746/798 kB Progress (1): 762/798 kB Progress (1): 779/798 kB Progress (1): 795/798 kB Progress (2): 795/798 kB | 16/30 kB Progress (2): 798 kB | 16/30 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/regions/2.20.43/regions-2.20.43.jar (798 kB at 4.8 MB/s) 2024-05-09T20:19:44.643 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 30 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar 2024-05-09T20:19:44.643 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/json-utils/2.20.43/json-utils-2.20.43.jar (30 kB at 180 kB/s) 2024-05-09T20:19:44.643 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar 2024-05-09T20:19:44.645 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/385 kB Progress (1): 33/385 kB Progress (1): 49/385 kB Progress (2): 49/385 kB | 13 kB Progress (2): 66/385 kB | 13 kB Progress (2): 82/385 kB | 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/endpoints-spi/2.20.43/endpoints-spi-2.20.43.jar (13 kB at 77 kB/s) 2024-05-09T20:19:44.645 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 98/385 kB Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar 2024-05-09T20:19:44.660 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 115/385 kB Progress (1): 131/385 kB Progress (1): 147/385 kB Progress (1): 164/385 kB Progress (1): 180/385 kB Progress (1): 197/385 kB Progress (1): 213/385 kB Progress (1): 229/385 kB Progress (1): 246/385 kB Progress (1): 262/385 kB Progress (1): 279/385 kB Progress (2): 279/385 kB | 16/75 kB Progress (2): 295/385 kB | 16/75 kB Progress (2): 311/385 kB | 16/75 kB Progress (2): 328/385 kB | 16/75 kB Progress (2): 344/385 kB | 16/75 kB Progress (2): 344/385 kB | 32/75 kB Progress (2): 344/385 kB | 49/75 kB Progress (2): 344/385 kB | 65/75 kB Progress (2): 360/385 kB | 65/75 kB Progress (2): 377/385 kB | 65/75 kB Progress (2): 385 kB | 65/75 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/third-party-jackson-core/2.20.43/third-party-jackson-core-2.20.43.jar (385 kB at 2.1 MB/s) 2024-05-09T20:19:44.660 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar 2024-05-09T20:19:44.668 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 65/75 kB | 16/329 kB Progress (2): 65/75 kB | 33/329 kB Progress (2): 65/75 kB | 49/329 kB Progress (2): 65/75 kB | 66/329 kB Progress (2): 65/75 kB | 82/329 kB Progress (2): 65/75 kB | 98/329 kB Progress (2): 65/75 kB | 115/329 kB Progress (2): 65/75 kB | 131/329 kB Progress (2): 65/75 kB | 147/329 kB Progress (3): 65/75 kB | 147/329 kB | 16/780 kB Progress (3): 65/75 kB | 164/329 kB | 16/780 kB Progress (3): 65/75 kB | 180/329 kB | 16/780 kB Progress (4): 65/75 kB | 180/329 kB | 16/780 kB | 16/501 kB Progress (4): 65/75 kB | 197/329 kB | 16/780 kB | 16/501 kB Progress (4): 65/75 kB | 213/329 kB | 16/780 kB | 16/501 kB Progress (4): 65/75 kB | 213/329 kB | 16/780 kB | 33/501 kB Progress (4): 65/75 kB | 229/329 kB | 16/780 kB | 33/501 kB Progress (4): 65/75 kB | 246/329 kB | 16/780 kB | 33/501 kB Progress (4): 65/75 kB | 246/329 kB | 33/780 kB | 33/501 kB Progress (4): 65/75 kB | 256/329 kB | 33/780 kB | 33/501 kB Progress (4): 65/75 kB | 256/329 kB | 33/780 kB | 49/501 kB Progress (4): 65/75 kB | 272/329 kB | 33/780 kB | 49/501 kB Progress (4): 65/75 kB | 289/329 kB | 33/780 kB | 49/501 kB Progress (4): 65/75 kB | 289/329 kB | 33/780 kB | 62/501 kB Progress (4): 65/75 kB | 305/329 kB | 33/780 kB | 62/501 kB Progress (4): 65/75 kB | 305/329 kB | 49/780 kB | 62/501 kB Progress (4): 65/75 kB | 321/329 kB | 49/780 kB | 62/501 kB Progress (4): 65/75 kB | 321/329 kB | 49/780 kB | 79/501 kB Progress (4): 75 kB | 321/329 kB | 49/780 kB | 79/501 kB Progress (4): 75 kB | 329 kB | 49/780 kB | 79/501 kB Progress (4): 75 kB | 329 kB | 66/780 kB | 79/501 kB Progress (4): 75 kB | 329 kB | 66/780 kB | 95/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/software/amazon/awssdk/apache-client/2.20.43/apache-client-2.20.43.jar (75 kB at 388 kB/s) 2024-05-09T20:19:44.668 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar (329 kB at 1.7 MB/s) 2024-05-09T20:19:44.669 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar 2024-05-09T20:19:44.669 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar 2024-05-09T20:19:44.685 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 82/780 kB | 95/501 kB Progress (2): 82/780 kB | 112/501 kB Progress (2): 98/780 kB | 112/501 kB Progress (2): 98/780 kB | 128/501 kB Progress (2): 98/780 kB | 144/501 kB Progress (2): 98/780 kB | 161/501 kB Progress (2): 115/780 kB | 161/501 kB Progress (2): 115/780 kB | 177/501 kB Progress (2): 115/780 kB | 193/501 kB Progress (2): 115/780 kB | 210/501 kB Progress (2): 131/780 kB | 210/501 kB Progress (2): 131/780 kB | 226/501 kB Progress (2): 131/780 kB | 243/501 kB Progress (2): 131/780 kB | 259/501 kB Progress (2): 147/780 kB | 259/501 kB Progress (2): 147/780 kB | 275/501 kB Progress (2): 147/780 kB | 292/501 kB Progress (2): 164/780 kB | 292/501 kB Progress (3): 164/780 kB | 292/501 kB | 16/354 kB Progress (3): 164/780 kB | 292/501 kB | 32/354 kB Progress (3): 164/780 kB | 308/501 kB | 32/354 kB Progress (3): 164/780 kB | 308/501 kB | 49/354 kB Progress (3): 164/780 kB | 312/501 kB | 49/354 kB Progress (3): 164/780 kB | 312/501 kB | 65/354 kB Progress (3): 164/780 kB | 324/501 kB | 65/354 kB Progress (3): 164/780 kB | 324/501 kB | 81/354 kB Progress (3): 164/780 kB | 341/501 kB | 81/354 kB Progress (3): 164/780 kB | 341/501 kB | 98/354 kB Progress (3): 164/780 kB | 341/501 kB | 114/354 kB Progress (3): 164/780 kB | 357/501 kB | 114/354 kB Progress (3): 164/780 kB | 357/501 kB | 130/354 kB Progress (3): 164/780 kB | 374/501 kB | 130/354 kB Progress (3): 164/780 kB | 374/501 kB | 147/354 kB Progress (3): 164/780 kB | 374/501 kB | 163/354 kB Progress (3): 164/780 kB | 390/501 kB | 163/354 kB Progress (3): 164/780 kB | 390/501 kB | 180/354 kB Progress (3): 164/780 kB | 406/501 kB | 180/354 kB Progress (3): 164/780 kB | 406/501 kB | 196/354 kB Progress (3): 180/780 kB | 406/501 kB | 196/354 kB Progress (3): 180/780 kB | 406/501 kB | 212/354 kB Progress (3): 180/780 kB | 423/501 kB | 212/354 kB Progress (3): 180/780 kB | 423/501 kB | 229/354 kB Progress (3): 180/780 kB | 439/501 kB | 229/354 kB Progress (3): 197/780 kB | 439/501 kB | 229/354 kB Progress (3): 197/780 kB | 439/501 kB | 245/354 kB Progress (3): 197/780 kB | 456/501 kB | 245/354 kB Progress (3): 197/780 kB | 456/501 kB | 262/354 kB Progress (3): 213/780 kB | 456/501 kB | 262/354 kB Progress (3): 213/780 kB | 472/501 kB | 262/354 kB Progress (3): 213/780 kB | 472/501 kB | 278/354 kB Progress (3): 213/780 kB | 488/501 kB | 278/354 kB Progress (3): 229/780 kB | 488/501 kB | 278/354 kB Progress (3): 229/780 kB | 501 kB | 278/354 kB Progress (3): 229/780 kB | 501 kB | 294/354 kB Progress (4): 229/780 kB | 501 kB | 294/354 kB | 6.4 kB Progress (4): 229/780 kB | 501 kB | 311/354 kB | 6.4 kB Progress (4): 246/780 kB | 501 kB | 311/354 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.1/commons-io-2.15.1.jar (501 kB at 2.4 MB/s) 2024-05-09T20:19:44.685 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar 2024-05-09T20:19:44.685 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 246/780 kB | 327/354 kB | 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter/5.10.1/junit-jupiter-5.10.1.jar (6.4 kB at 30 kB/s) 2024-05-09T20:19:44.685 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar 2024-05-09T20:19:44.687 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 246/780 kB | 343/354 kB Progress (3): 246/780 kB | 343/354 kB | 16/41 kB Progress (3): 246/780 kB | 354 kB | 16/41 kB Progress (3): 262/780 kB | 354 kB | 16/41 kB Progress (3): 262/780 kB | 354 kB | 32/41 kB Progress (3): 262/780 kB | 354 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.15/commons-codec-1.15.jar (354 kB at 1.7 MB/s) 2024-05-09T20:19:44.687 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar 2024-05-09T20:19:44.687 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 279/780 kB | 41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar (41 kB at 195 kB/s) 2024-05-09T20:19:44.687 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar 2024-05-09T20:19:44.704 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 295/780 kB Progress (1): 311/780 kB Progress (1): 328/780 kB Progress (1): 344/780 kB Progress (1): 360/780 kB Progress (1): 377/780 kB Progress (1): 393/780 kB Progress (1): 410/780 kB Progress (1): 426/780 kB Progress (2): 426/780 kB | 16/211 kB Progress (3): 426/780 kB | 16/211 kB | 14 kB Progress (3): 426/780 kB | 32/211 kB | 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar (14 kB at 63 kB/s) 2024-05-09T20:19:44.705 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 426/780 kB | 49/211 kB Progress (3): 426/780 kB | 49/211 kB | 16/586 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar 2024-05-09T20:19:44.709 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 426/780 kB | 65/211 kB | 16/586 kB Progress (3): 426/780 kB | 65/211 kB | 33/586 kB Progress (3): 426/780 kB | 81/211 kB | 33/586 kB Progress (3): 426/780 kB | 98/211 kB | 33/586 kB Progress (3): 426/780 kB | 98/211 kB | 49/586 kB Progress (3): 426/780 kB | 114/211 kB | 49/586 kB Progress (3): 426/780 kB | 114/211 kB | 66/586 kB Progress (3): 426/780 kB | 130/211 kB | 66/586 kB Progress (3): 426/780 kB | 130/211 kB | 82/586 kB Progress (3): 426/780 kB | 130/211 kB | 98/586 kB Progress (3): 442/780 kB | 130/211 kB | 98/586 kB Progress (3): 442/780 kB | 147/211 kB | 98/586 kB Progress (3): 442/780 kB | 147/211 kB | 115/586 kB Progress (3): 442/780 kB | 163/211 kB | 115/586 kB Progress (3): 442/780 kB | 180/211 kB | 115/586 kB Progress (3): 442/780 kB | 180/211 kB | 131/586 kB Progress (4): 442/780 kB | 180/211 kB | 131/586 kB | 16/106 kB Progress (4): 442/780 kB | 196/211 kB | 131/586 kB | 16/106 kB Progress (4): 442/780 kB | 211 kB | 131/586 kB | 16/106 kB Progress (4): 442/780 kB | 211 kB | 147/586 kB | 16/106 kB Progress (4): 442/780 kB | 211 kB | 147/586 kB | 33/106 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-api/5.10.1/junit-jupiter-api-5.10.1.jar (211 kB at 909 kB/s) 2024-05-09T20:19:44.709 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar 2024-05-09T20:19:44.712 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 442/780 kB | 164/586 kB | 33/106 kB Progress (3): 442/780 kB | 164/586 kB | 49/106 kB Progress (3): 442/780 kB | 180/586 kB | 49/106 kB Progress (3): 442/780 kB | 180/586 kB | 64/106 kB Progress (3): 442/780 kB | 197/586 kB | 64/106 kB Progress (3): 442/780 kB | 197/586 kB | 81/106 kB Progress (3): 459/780 kB | 197/586 kB | 81/106 kB Progress (3): 459/780 kB | 213/586 kB | 81/106 kB Progress (3): 459/780 kB | 213/586 kB | 97/106 kB Progress (3): 459/780 kB | 213/586 kB | 106 kB Progress (3): 459/780 kB | 229/586 kB | 106 kB Progress (3): 475/780 kB | 229/586 kB | 106 kB Progress (3): 475/780 kB | 246/586 kB | 106 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.10.1/junit-platform-commons-1.10.1.jar (106 kB at 450 kB/s) 2024-05-09T20:19:44.712 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar 2024-05-09T20:19:44.721 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 475/780 kB | 256/586 kB Progress (2): 492/780 kB | 256/586 kB Progress (2): 492/780 kB | 272/586 kB Progress (2): 492/780 kB | 289/586 kB Progress (2): 492/780 kB | 305/586 kB Progress (2): 508/780 kB | 305/586 kB Progress (2): 508/780 kB | 321/586 kB Progress (2): 524/780 kB | 321/586 kB Progress (2): 524/780 kB | 338/586 kB Progress (2): 541/780 kB | 338/586 kB Progress (2): 541/780 kB | 354/586 kB Progress (2): 541/780 kB | 370/586 kB Progress (2): 557/780 kB | 370/586 kB Progress (2): 557/780 kB | 387/586 kB Progress (2): 557/780 kB | 403/586 kB Progress (2): 557/780 kB | 420/586 kB Progress (2): 557/780 kB | 436/586 kB Progress (2): 573/780 kB | 436/586 kB Progress (2): 573/780 kB | 452/586 kB Progress (2): 573/780 kB | 469/586 kB Progress (2): 585/780 kB | 469/586 kB Progress (2): 585/780 kB | 485/586 kB Progress (2): 585/780 kB | 502/586 kB Progress (2): 585/780 kB | 518/586 kB Progress (2): 585/780 kB | 534/586 kB Progress (2): 585/780 kB | 551/586 kB Progress (2): 601/780 kB | 551/586 kB Progress (2): 601/780 kB | 567/586 kB Progress (2): 601/780 kB | 583/586 kB Progress (2): 601/780 kB | 586 kB Progress (3): 601/780 kB | 586 kB | 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apiguardian/apiguardian-api/1.1.2/apiguardian-api-1.1.2.jar (6.8 kB at 28 kB/s) 2024-05-09T20:19:44.721 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-params/5.10.1/junit-jupiter-params-5.10.1.jar (586 kB at 2.4 MB/s) 2024-05-09T20:19:44.722 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar 2024-05-09T20:19:44.722 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar 2024-05-09T20:19:44.733 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 617/780 kB Progress (2): 617/780 kB | 16/245 kB Progress (2): 634/780 kB | 16/245 kB Progress (2): 634/780 kB | 33/245 kB Progress (2): 634/780 kB | 49/245 kB Progress (2): 634/780 kB | 66/245 kB Progress (2): 650/780 kB | 66/245 kB Progress (2): 650/780 kB | 82/245 kB Progress (2): 650/780 kB | 98/245 kB Progress (2): 650/780 kB | 115/245 kB Progress (2): 650/780 kB | 131/245 kB Progress (2): 650/780 kB | 147/245 kB Progress (3): 650/780 kB | 147/245 kB | 16/205 kB Progress (3): 650/780 kB | 164/245 kB | 16/205 kB Progress (3): 667/780 kB | 164/245 kB | 16/205 kB Progress (3): 667/780 kB | 164/245 kB | 32/205 kB Progress (3): 667/780 kB | 180/245 kB | 32/205 kB Progress (3): 667/780 kB | 180/245 kB | 49/205 kB Progress (3): 683/780 kB | 180/245 kB | 49/205 kB Progress (3): 683/780 kB | 197/245 kB | 49/205 kB Progress (3): 683/780 kB | 197/245 kB | 65/205 kB Progress (3): 683/780 kB | 213/245 kB | 65/205 kB Progress (3): 683/780 kB | 213/245 kB | 81/205 kB Progress (3): 683/780 kB | 229/245 kB | 81/205 kB Progress (3): 683/780 kB | 229/245 kB | 98/205 kB Progress (3): 683/780 kB | 245 kB | 98/205 kB Progress (3): 683/780 kB | 245 kB | 114/205 kB Progress (3): 699/780 kB | 245 kB | 114/205 kB Progress (3): 699/780 kB | 245 kB | 131/205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/jupiter/junit-jupiter-engine/5.10.1/junit-jupiter-engine-5.10.1.jar (245 kB at 952 kB/s) 2024-05-09T20:19:44.733 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar 2024-05-09T20:19:44.735 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 699/780 kB | 147/205 kB Progress (2): 699/780 kB | 163/205 kB Progress (2): 699/780 kB | 180/205 kB Progress (2): 716/780 kB | 180/205 kB Progress (2): 716/780 kB | 196/205 kB Progress (2): 716/780 kB | 205 kB Progress (2): 732/780 kB | 205 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.10.1/junit-platform-engine-1.10.1.jar (205 kB at 791 kB/s) 2024-05-09T20:19:44.735 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar 2024-05-09T20:19:44.739 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 749/780 kB Progress (1): 765/780 kB Progress (2): 765/780 kB | 15 kB Progress (3): 765/780 kB | 15 kB | 16/16 kB Progress (3): 765/780 kB | 15 kB | 16 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-simple/1.7.28/slf4j-simple-1.7.28.jar (15 kB at 58 kB/s) 2024-05-09T20:19:44.739 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar 2024-05-09T20:19:44.739 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/jcl-over-slf4j/1.7.28/jcl-over-slf4j-1.7.28.jar (16 kB at 63 kB/s) 2024-05-09T20:19:44.739 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar 2024-05-09T20:19:44.741 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 780 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar (780 kB at 2.9 MB/s) 2024-05-09T20:19:44.741 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar 2024-05-09T20:19:44.750 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite/1.10.1/junit-platform-suite-1.10.1.jar (6.4 kB at 23 kB/s) 2024-05-09T20:19:44.755 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-api/1.10.1/junit-platform-suite-api-1.10.1.jar (23 kB at 81 kB/s) 2024-05-09T20:19:44.756 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/24 kB Progress (1): 24 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-engine/1.10.1/junit-platform-suite-engine-1.10.1.jar (24 kB at 86 kB/s) 2024-05-09T20:19:44.757 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-suite-commons/1.10.1/junit-platform-suite-commons-1.10.1.jar (17 kB at 62 kB/s) 2024-05-09T20:19:44.774 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/184 kB Progress (1): 33/184 kB Progress (1): 49/184 kB Progress (1): 66/184 kB Progress (1): 82/184 kB Progress (1): 98/184 kB Progress (1): 115/184 kB Progress (1): 131/184 kB Progress (1): 147/184 kB Progress (1): 164/184 kB Progress (1): 180/184 kB Progress (1): 184 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.10.1/junit-platform-launcher-1.10.1.jar (184 kB at 617 kB/s) 2024-05-09T20:19:44.800 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:44.800 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- clean:3.2.0:clean (default-clean) @ jcksum --- 2024-05-09T20:19:44.810 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom 2024-05-09T20:19:44.829 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.pom (5.8 kB at 307 kB/s) 2024-05-09T20:19:44.833 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom 2024-05-09T20:19:44.852 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/34/maven-shared-components-34.pom (5.1 kB at 268 kB/s) 2024-05-09T20:19:44.855 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom 2024-05-09T20:19:44.883 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/43 kB Progress (1): 33/43 kB Progress (1): 43 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 1.5 MB/s) 2024-05-09T20:19:44.891 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom 2024-05-09T20:19:44.924 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.pom (14 kB at 432 kB/s) 2024-05-09T20:19:44.934 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar 2024-05-09T20:19:44.969 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/153 kB Progress (1): 33/153 kB Progress (1): 49/153 kB Progress (1): 66/153 kB Progress (1): 82/153 kB Progress (1): 98/153 kB Progress (1): 115/153 kB Progress (1): 131/153 kB Progress (1): 147/153 kB Progress (1): 153 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar (153 kB at 4.3 MB/s) 2024-05-09T20:19:44.970 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar 2024-05-09T20:19:45.006 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/215 kB Progress (1): 33/215 kB Progress (1): 49/215 kB Progress (1): 66/215 kB Progress (1): 82/215 kB Progress (1): 98/215 kB Progress (1): 115/215 kB Progress (1): 131/215 kB Progress (1): 147/215 kB Progress (1): 164/215 kB Progress (1): 180/215 kB Progress (1): 197/215 kB Progress (1): 213/215 kB Progress (1): 215 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.6/commons-io-2.6.jar (215 kB at 5.8 MB/s) 2024-05-09T20:19:45.114 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:45.114 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-05-09T20:19:45.119 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom 2024-05-09T20:19:45.137 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.pom (2.7 kB at 148 kB/s) 2024-05-09T20:19:45.139 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom 2024-05-09T20:19:45.158 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/5.1/plexus-5.1.pom (23 kB at 1.2 MB/s) 2024-05-09T20:19:45.162 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom 2024-05-09T20:19:45.180 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.pom (8.8 kB at 487 kB/s) 2024-05-09T20:19:45.184 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom 2024-05-09T20:19:45.203 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/10/plexus-10.pom (25 kB at 1.3 MB/s) 2024-05-09T20:19:45.208 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom 2024-05-09T20:19:45.226 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.pom (6.0 kB at 318 kB/s) 2024-05-09T20:19:45.229 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom 2024-05-09T20:19:45.247 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/39/maven-shared-components-39.pom (3.2 kB at 179 kB/s) 2024-05-09T20:19:45.252 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom 2024-05-09T20:19:45.270 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 612 B Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.pom (612 B at 34 kB/s) 2024-05-09T20:19:45.273 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom 2024-05-09T20:19:45.291 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.pom (2.7 kB at 152 kB/s) 2024-05-09T20:19:45.293 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom 2024-05-09T20:19:45.312 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-parent/1.7.36/slf4j-parent-1.7.36.pom (14 kB at 742 kB/s) 2024-05-09T20:19:45.316 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom 2024-05-09T20:19:45.334 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.pom (3.2 kB at 178 kB/s) 2024-05-09T20:19:45.337 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom 2024-05-09T20:19:45.355 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/15/spice-parent-15.pom (8.4 kB at 464 kB/s) 2024-05-09T20:19:45.357 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom 2024-05-09T20:19:45.375 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/5/forge-parent-5.pom (8.4 kB at 465 kB/s) 2024-05-09T20:19:45.379 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom 2024-05-09T20:19:45.398 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.pom (8.0 kB at 422 kB/s) 2024-05-09T20:19:45.403 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom 2024-05-09T20:19:45.422 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.pom (20 kB at 1.0 MB/s) 2024-05-09T20:19:45.430 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom 2024-05-09T20:19:45.447 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.2/junit-bom-5.7.2.pom (5.1 kB at 283 kB/s) 2024-05-09T20:19:45.451 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom 2024-05-09T20:19:45.470 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/31 kB Progress (1): 31 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.pom (31 kB at 1.6 MB/s) 2024-05-09T20:19:45.477 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom 2024-05-09T20:19:45.495 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.7.1/junit-bom-5.7.1.pom (5.1 kB at 283 kB/s) 2024-05-09T20:19:45.501 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar 2024-05-09T20:19:45.525 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/85 kB Progress (1): 33/85 kB Progress (1): 49/85 kB Progress (1): 66/85 kB Progress (1): 82/85 kB Progress (1): 85 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar (85 kB at 3.6 MB/s) 2024-05-09T20:19:45.525 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar 2024-05-09T20:19:45.525 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar 2024-05-09T20:19:45.526 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar 2024-05-09T20:19:45.526 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar 2024-05-09T20:19:45.526 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar 2024-05-09T20:19:45.546 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.5 kB Progress (2): 2.5 kB | 8.5 kB Progress (3): 2.5 kB | 8.5 kB | 16/41 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/inject/javax.inject/1/javax.inject-1.jar (2.5 kB at 131 kB/s) 2024-05-09T20:19:45.546 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 8.5 kB | 33/41 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-build-api/0.0.7/plexus-build-api-0.0.7.jar (8.5 kB at 446 kB/s) 2024-05-09T20:19:45.546 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 33/41 kB | 16/269 kB Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar 2024-05-09T20:19:45.546 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 41 kB | 16/269 kB Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar 2024-05-09T20:19:45.546 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 41 kB | 16/269 kB | 16/55 kB Progress (3): 41 kB | 33/269 kB | 16/55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/slf4j/slf4j-api/1.7.36/slf4j-api-1.7.36.jar (41 kB at 2.1 MB/s) 2024-05-09T20:19:45.547 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 33/269 kB | 33/55 kB Progress (2): 49/269 kB | 33/55 kB Progress (2): 49/269 kB | 49/55 kB Progress (2): 49/269 kB | 55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-filtering/3.3.1/maven-filtering-3.3.1.jar (55 kB at 2.5 MB/s) 2024-05-09T20:19:45.575 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 63/269 kB Progress (1): 79/269 kB Progress (1): 95/269 kB Progress (1): 112/269 kB Progress (1): 128/269 kB Progress (1): 145/269 kB Progress (1): 161/269 kB Progress (1): 177/269 kB Progress (1): 194/269 kB Progress (1): 210/269 kB Progress (2): 210/269 kB | 16/327 kB Progress (2): 210/269 kB | 33/327 kB Progress (2): 210/269 kB | 49/327 kB Progress (3): 210/269 kB | 49/327 kB | 16/587 kB Progress (3): 210/269 kB | 62/327 kB | 16/587 kB Progress (3): 210/269 kB | 62/327 kB | 33/587 kB Progress (3): 210/269 kB | 78/327 kB | 33/587 kB Progress (3): 226/269 kB | 78/327 kB | 33/587 kB Progress (3): 226/269 kB | 78/327 kB | 49/587 kB Progress (3): 226/269 kB | 78/327 kB | 66/587 kB Progress (3): 226/269 kB | 95/327 kB | 66/587 kB Progress (3): 226/269 kB | 95/327 kB | 82/587 kB Progress (3): 226/269 kB | 95/327 kB | 98/587 kB Progress (3): 226/269 kB | 111/327 kB | 98/587 kB Progress (3): 226/269 kB | 111/327 kB | 115/587 kB Progress (3): 226/269 kB | 128/327 kB | 115/587 kB Progress (3): 226/269 kB | 128/327 kB | 131/587 kB Progress (3): 243/269 kB | 128/327 kB | 131/587 kB Progress (3): 243/269 kB | 144/327 kB | 131/587 kB Progress (3): 243/269 kB | 144/327 kB | 147/587 kB Progress (3): 243/269 kB | 144/327 kB | 164/587 kB Progress (3): 243/269 kB | 160/327 kB | 164/587 kB Progress (3): 243/269 kB | 160/327 kB | 180/587 kB Progress (3): 243/269 kB | 177/327 kB | 180/587 kB Progress (3): 243/269 kB | 177/327 kB | 197/587 kB Progress (3): 259/269 kB | 177/327 kB | 197/587 kB Progress (3): 259/269 kB | 193/327 kB | 197/587 kB Progress (3): 259/269 kB | 193/327 kB | 213/587 kB Progress (3): 259/269 kB | 193/327 kB | 229/587 kB Progress (3): 259/269 kB | 210/327 kB | 229/587 kB Progress (3): 269 kB | 210/327 kB | 229/587 kB Progress (3): 269 kB | 210/327 kB | 246/587 kB Progress (3): 269 kB | 226/327 kB | 246/587 kB Progress (3): 269 kB | 226/327 kB | 256/587 kB Progress (3): 269 kB | 242/327 kB | 256/587 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar (269 kB at 5.5 MB/s) 2024-05-09T20:19:45.579 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 242/327 kB | 272/587 kB Progress (2): 259/327 kB | 272/587 kB Progress (2): 259/327 kB | 289/587 kB Progress (2): 259/327 kB | 305/587 kB Progress (2): 275/327 kB | 305/587 kB Progress (2): 275/327 kB | 321/587 kB Progress (2): 291/327 kB | 321/587 kB Progress (2): 291/327 kB | 338/587 kB Progress (2): 291/327 kB | 354/587 kB Progress (2): 308/327 kB | 354/587 kB Progress (2): 308/327 kB | 370/587 kB Progress (2): 324/327 kB | 370/587 kB Progress (2): 324/327 kB | 387/587 kB Progress (2): 327 kB | 387/587 kB Progress (2): 327 kB | 403/587 kB Progress (2): 327 kB | 420/587 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.11.0/commons-io-2.11.0.jar (327 kB at 6.2 MB/s) 2024-05-09T20:19:45.583 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 436/587 kB Progress (1): 452/587 kB Progress (1): 469/587 kB Progress (1): 485/587 kB Progress (1): 502/587 kB Progress (1): 518/587 kB Progress (1): 534/587 kB Progress (1): 551/587 kB Progress (1): 567/587 kB Progress (1): 583/587 kB Progress (1): 587 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar (587 kB at 10 MB/s) 2024-05-09T20:19:45.630 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-05-09T20:19:45.631 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:45.631 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-05-09T20:19:45.637 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom 2024-05-09T20:19:45.655 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.pom (4.7 kB at 263 kB/s) 2024-05-09T20:19:45.658 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom 2024-05-09T20:19:45.676 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/19/maven-shared-components-19.pom (6.4 kB at 353 kB/s) 2024-05-09T20:19:45.679 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom 2024-05-09T20:19:45.696 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/33 kB Progress (1): 33 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 1.9 MB/s) 2024-05-09T20:19:45.699 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2024-05-09T20:19:45.717 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 776 kB/s) 2024-05-09T20:19:45.721 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom 2024-05-09T20:19:45.739 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 815 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.pom (815 B at 45 kB/s) 2024-05-09T20:19:45.742 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom 2024-05-09T20:19:45.760 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/1.5.5/plexus-containers-1.5.5.pom (4.2 kB at 236 kB/s) 2024-05-09T20:19:45.762 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom 2024-05-09T20:19:45.784 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/2.0.7/plexus-2.0.7.pom (17 kB at 786 kB/s) 2024-05-09T20:19:45.791 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom 2024-05-09T20:19:45.811 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.pom (5.0 kB at 249 kB/s) 2024-05-09T20:19:45.815 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom 2024-05-09T20:19:45.834 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.1.2/plexus-languages-1.1.2.pom (4.1 kB at 230 kB/s) 2024-05-09T20:19:45.839 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom 2024-05-09T20:19:45.882 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.pom (2.4 kB at 55 kB/s) 2024-05-09T20:19:45.886 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom 2024-05-09T20:19:45.928 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/ow2/1.5.1/ow2-1.5.1.pom (11 kB at 275 kB/s) 2024-05-09T20:19:45.932 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom 2024-05-09T20:19:45.953 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.pom (17 kB at 821 kB/s) 2024-05-09T20:19:45.958 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2024-05-09T20:19:45.976 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom (6.6 kB at 346 kB/s) 2024-05-09T20:19:45.981 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom 2024-05-09T20:19:46.000 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.pom (1.1 kB at 56 kB/s) 2024-05-09T20:19:46.004 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom 2024-05-09T20:19:46.025 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler/2.13.0/plexus-compiler-2.13.0.pom (8.4 kB at 399 kB/s) 2024-05-09T20:19:46.029 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom 2024-05-09T20:19:46.047 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/10.0/plexus-components-10.0.pom (2.7 kB at 149 kB/s) 2024-05-09T20:19:46.053 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom 2024-05-09T20:19:46.072 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.9.1/junit-bom-5.9.1.pom (5.6 kB at 296 kB/s) 2024-05-09T20:19:46.078 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom 2024-05-09T20:19:46.097 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.pom (1.1 kB at 60 kB/s) 2024-05-09T20:19:46.104 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom 2024-05-09T20:19:46.123 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 770 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.1/plexus-component-annotations-2.1.1.pom (770 B at 41 kB/s) 2024-05-09T20:19:46.127 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom 2024-05-09T20:19:46.146 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.1/plexus-containers-2.1.1.pom (6.0 kB at 318 kB/s) 2024-05-09T20:19:46.150 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom 2024-05-09T20:19:46.169 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/26 kB Progress (1): 26 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/6.5/plexus-6.5.pom (26 kB at 1.4 MB/s) 2024-05-09T20:19:46.175 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom 2024-05-09T20:19:46.193 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.pom (1.2 kB at 67 kB/s) 2024-05-09T20:19:46.196 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom 2024-05-09T20:19:46.215 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compilers/2.13.0/plexus-compilers-2.13.0.pom (1.3 kB at 70 kB/s) 2024-05-09T20:19:46.229 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar 2024-05-09T20:19:46.249 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-incremental/1.1/maven-shared-incremental-1.1.jar (14 kB at 677 kB/s) 2024-05-09T20:19:46.249 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar 2024-05-09T20:19:46.250 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar 2024-05-09T20:19:46.250 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar 2024-05-09T20:19:46.250 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar 2024-05-09T20:19:46.251 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar 2024-05-09T20:19:46.268 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.2 kB Progress (2): 4.2 kB | 16/55 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar (4.2 kB at 222 kB/s) 2024-05-09T20:19:46.269 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar 2024-05-09T20:19:46.271 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 33/55 kB Progress (1): 49/55 kB Progress (1): 55 kB Progress (2): 55 kB | 16/122 kB Progress (2): 55 kB | 33/122 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.1.2/plexus-java-1.1.2.jar (55 kB at 2.6 MB/s) 2024-05-09T20:19:46.271 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar 2024-05-09T20:19:46.272 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 49/122 kB Progress (2): 49/122 kB | 16/27 kB Progress (2): 49/122 kB | 27 kB Progress (2): 66/122 kB | 27 kB Progress (3): 66/122 kB | 27 kB | 16/334 kB Progress (3): 82/122 kB | 27 kB | 16/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-api/2.13.0/plexus-compiler-api-2.13.0.jar (27 kB at 1.3 MB/s) 2024-05-09T20:19:46.272 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar 2024-05-09T20:19:46.273 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 98/122 kB | 16/334 kB Progress (2): 115/122 kB | 16/334 kB Progress (2): 122 kB | 16/334 kB Progress (2): 122 kB | 33/334 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.4/asm-9.4.jar (122 kB at 5.3 MB/s) 2024-05-09T20:19:46.290 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 49/334 kB Progress (1): 66/334 kB Progress (1): 82/334 kB Progress (1): 98/334 kB Progress (1): 115/334 kB Progress (1): 131/334 kB Progress (1): 147/334 kB Progress (1): 164/334 kB Progress (2): 164/334 kB | 16/267 kB Progress (2): 164/334 kB | 33/267 kB Progress (2): 164/334 kB | 49/267 kB Progress (2): 164/334 kB | 66/267 kB Progress (2): 164/334 kB | 82/267 kB Progress (2): 180/334 kB | 82/267 kB Progress (2): 180/334 kB | 98/267 kB Progress (3): 180/334 kB | 98/267 kB | 4.7 kB Progress (4): 180/334 kB | 98/267 kB | 4.7 kB | 16/23 kB Progress (4): 197/334 kB | 98/267 kB | 4.7 kB | 16/23 kB Progress (4): 197/334 kB | 115/267 kB | 4.7 kB | 16/23 kB Progress (4): 197/334 kB | 115/267 kB | 4.7 kB | 23 kB Progress (4): 197/334 kB | 131/267 kB | 4.7 kB | 23 kB Progress (4): 213/334 kB | 131/267 kB | 4.7 kB | 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-manager/2.13.0/plexus-compiler-manager-2.13.0.jar (4.7 kB at 123 kB/s) 2024-05-09T20:19:46.290 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-compiler-javac/2.13.0/plexus-compiler-javac-2.13.0.jar (23 kB at 584 kB/s) 2024-05-09T20:19:46.293 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 213/334 kB | 147/267 kB Progress (2): 213/334 kB | 164/267 kB Progress (2): 229/334 kB | 164/267 kB Progress (2): 229/334 kB | 180/267 kB Progress (2): 229/334 kB | 197/267 kB Progress (2): 246/334 kB | 197/267 kB Progress (2): 246/334 kB | 213/267 kB Progress (2): 246/334 kB | 229/267 kB Progress (2): 246/334 kB | 246/267 kB Progress (2): 246/334 kB | 262/267 kB Progress (2): 246/334 kB | 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.5.0/plexus-utils-3.5.0.jar (267 kB at 6.4 MB/s) 2024-05-09T20:19:46.301 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 262/334 kB Progress (1): 279/334 kB Progress (1): 295/334 kB Progress (1): 311/334 kB Progress (1): 328/334 kB Progress (1): 334 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/thoughtworks/qdox/qdox/2.0.3/qdox-2.0.3.jar (334 kB at 6.6 MB/s) 2024-05-09T20:19:46.463 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Changes detected - recompiling the module! :source 2024-05-09T20:19:46.465 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/classes 2024-05-09T20:19:47.714 INFO:tasks.workunit.client.0.smithi063.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-05-09T20:19:47.714 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:47.714 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-05-09T20:19:47.719 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-05-09T20:19:47.719 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:47.719 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-05-09T20:19:47.727 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Changes detected - recompiling the module! :dependency 2024-05-09T20:19:47.728 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Compiling 1 source file with javac [debug target 1.9] to target/test-classes 2024-05-09T20:19:47.907 INFO:tasks.workunit.client.0.smithi063.stdout:[WARNING] bootstrap class path not set in conjunction with -source 9 2024-05-09T20:19:47.907 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:47.907 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-05-09T20:19:47.913 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom 2024-05-09T20:19:47.931 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.pom (6.2 kB at 329 kB/s) 2024-05-09T20:19:47.935 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom 2024-05-09T20:19:47.953 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.pom (3.5 kB at 196 kB/s) 2024-05-09T20:19:47.957 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom 2024-05-09T20:19:47.982 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.pom (3.3 kB at 125 kB/s) 2024-05-09T20:19:47.989 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom 2024-05-09T20:19:48.010 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.pom (4.1 kB at 176 kB/s) 2024-05-09T20:19:48.015 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom 2024-05-09T20:19:48.037 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.pom (3.3 kB at 150 kB/s) 2024-05-09T20:19:48.043 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom 2024-05-09T20:19:48.061 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.pom (4.5 kB at 248 kB/s) 2024-05-09T20:19:48.070 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom 2024-05-09T20:19:48.086 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.pom (1.8 kB at 93 kB/s) 2024-05-09T20:19:48.091 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom 2024-05-09T20:19:48.109 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.pom (2.2 kB at 122 kB/s) 2024-05-09T20:19:48.111 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom 2024-05-09T20:19:48.130 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/30 kB Progress (1): 30 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether/1.0.0.v20140518/aether-1.0.0.v20140518.pom (30 kB at 1.6 MB/s) 2024-05-09T20:19:48.134 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom 2024-05-09T20:19:48.153 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.pom (1.9 kB at 100 kB/s) 2024-05-09T20:19:48.157 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom 2024-05-09T20:19:48.175 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.pom (5.8 kB at 322 kB/s) 2024-05-09T20:19:48.180 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom 2024-05-09T20:19:48.200 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.2.5/maven-artifact-3.2.5.pom (2.3 kB at 117 kB/s) 2024-05-09T20:19:48.202 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom 2024-05-09T20:19:48.223 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven/3.2.5/maven-3.2.5.pom (22 kB at 1.1 MB/s) 2024-05-09T20:19:48.226 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom 2024-05-09T20:19:48.246 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/37 kB Progress (1): 33/37 kB Progress (1): 37 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/25/maven-parent-25.pom (37 kB at 1.9 MB/s) 2024-05-09T20:19:48.249 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2024-05-09T20:19:48.268 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom (15 kB at 802 kB/s) 2024-05-09T20:19:48.276 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom 2024-05-09T20:19:48.294 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/4.0.0/plexus-utils-4.0.0.pom (8.7 kB at 456 kB/s) 2024-05-09T20:19:48.298 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom 2024-05-09T20:19:48.317 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/13/plexus-13.pom (27 kB at 1.4 MB/s) 2024-05-09T20:19:48.323 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom 2024-05-09T20:19:48.343 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-core/3.2.5/maven-core-3.2.5.pom (8.1 kB at 403 kB/s) 2024-05-09T20:19:48.349 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom 2024-05-09T20:19:48.368 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings/3.2.5/maven-settings-3.2.5.pom (2.2 kB at 114 kB/s) 2024-05-09T20:19:48.374 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom 2024-05-09T20:19:48.394 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-settings-builder/3.2.5/maven-settings-builder-3.2.5.pom (2.6 kB at 130 kB/s) 2024-05-09T20:19:48.400 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom 2024-05-09T20:19:48.418 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-interpolation/1.21/plexus-interpolation-1.21.pom (1.5 kB at 86 kB/s) 2024-05-09T20:19:48.421 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom 2024-05-09T20:19:48.440 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-components/1.3.1/plexus-components-1.3.1.pom (3.1 kB at 161 kB/s) 2024-05-09T20:19:48.444 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom 2024-05-09T20:19:48.466 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/3.3.1/plexus-3.3.1.pom (20 kB at 929 kB/s) 2024-05-09T20:19:48.470 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom 2024-05-09T20:19:48.489 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/17/spice-parent-17.pom (6.8 kB at 356 kB/s) 2024-05-09T20:19:48.493 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom 2024-05-09T20:19:48.512 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 14 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/10/forge-parent-10.pom (14 kB at 714 kB/s) 2024-05-09T20:19:48.518 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom 2024-05-09T20:19:48.536 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-sec-dispatcher/1.3/plexus-sec-dispatcher-1.3.pom (3.0 kB at 164 kB/s) 2024-05-09T20:19:48.540 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom 2024-05-09T20:19:48.559 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/spice/spice-parent/12/spice-parent-12.pom (6.8 kB at 358 kB/s) 2024-05-09T20:19:48.563 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom 2024-05-09T20:19:48.585 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/4/forge-parent-4.pom (8.4 kB at 382 kB/s) 2024-05-09T20:19:48.590 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom 2024-05-09T20:19:48.614 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/plexus/plexus-cipher/1.4/plexus-cipher-1.4.pom (2.1 kB at 86 kB/s) 2024-05-09T20:19:48.619 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom 2024-05-09T20:19:48.641 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-repository-metadata/3.2.5/maven-repository-metadata-3.2.5.pom (2.2 kB at 101 kB/s) 2024-05-09T20:19:48.649 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom 2024-05-09T20:19:48.670 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-plugin-api/3.2.5/maven-plugin-api-3.2.5.pom (3.0 kB at 138 kB/s) 2024-05-09T20:19:48.678 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom 2024-05-09T20:19:48.699 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.plexus/0.9.0.M2/org.eclipse.sisu.plexus-0.9.0.M2.pom (15 kB at 680 kB/s) 2024-05-09T20:19:48.703 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom 2024-05-09T20:19:48.722 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-plexus/0.9.0.M2/sisu-plexus-0.9.0.M2.pom (15 kB at 763 kB/s) 2024-05-09T20:19:48.728 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom 2024-05-09T20:19:48.749 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/annotation/javax.annotation-api/1.2/javax.annotation-api-1.2.pom (13 kB at 672 kB/s) 2024-05-09T20:19:48.753 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom 2024-05-09T20:19:48.772 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/net/java/jvnet-parent/3/jvnet-parent-3.pom (4.8 kB at 252 kB/s) 2024-05-09T20:19:48.776 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom 2024-05-09T20:19:48.797 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/javax/enterprise/cdi-api/1.2/cdi-api-1.2.pom (6.3 kB at 299 kB/s) 2024-05-09T20:19:48.801 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom 2024-05-09T20:19:48.827 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/32 kB Progress (1): 32/32 kB Progress (1): 32 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/jboss/weld/weld-parent/26/weld-parent-26.pom (32 kB at 1.2 MB/s) 2024-05-09T20:19:48.834 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom 2024-05-09T20:19:48.862 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/17 kB Progress (1): 17 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/org.eclipse.sisu.inject/0.9.0.M2/org.eclipse.sisu.inject-0.9.0.M2.pom (17 kB at 619 kB/s) 2024-05-09T20:19:48.866 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom 2024-05-09T20:19:48.893 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/sisu/sisu-inject/0.9.0.M2/sisu-inject-0.9.0.M2.pom (15 kB at 548 kB/s) 2024-05-09T20:19:48.899 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom 2024-05-09T20:19:48.918 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 750 B Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.pom (750 B at 39 kB/s) 2024-05-09T20:19:48.921 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom 2024-05-09T20:19:48.940 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-containers/2.1.0/plexus-containers-2.1.0.pom (4.8 kB at 253 kB/s) 2024-05-09T20:19:48.945 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom 2024-05-09T20:19:48.966 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 7.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.6.0/plexus-classworlds-2.6.0.pom (7.9 kB at 396 kB/s) 2024-05-09T20:19:48.971 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom 2024-05-09T20:19:48.989 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-model-builder/3.2.5/maven-model-builder-3.2.5.pom (3.0 kB at 166 kB/s) 2024-05-09T20:19:48.996 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom 2024-05-09T20:19:49.014 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-aether-provider/3.2.5/maven-aether-provider-3.2.5.pom (4.2 kB at 236 kB/s) 2024-05-09T20:19:49.022 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom 2024-05-09T20:19:49.042 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-spi/1.0.0.v20140518/aether-spi-1.0.0.v20140518.pom (2.1 kB at 103 kB/s) 2024-05-09T20:19:49.048 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom 2024-05-09T20:19:49.069 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-impl/1.0.0.v20140518/aether-impl-1.0.0.v20140518.pom (3.5 kB at 165 kB/s) 2024-05-09T20:19:49.076 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom 2024-05-09T20:19:49.095 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 11 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/sisu-guice/3.2.3/sisu-guice-3.2.3.pom (11 kB at 547 kB/s) 2024-05-09T20:19:49.099 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom 2024-05-09T20:19:49.121 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 13 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/sisu/inject/guice-parent/3.2.3/guice-parent-3.2.3.pom (13 kB at 612 kB/s) 2024-05-09T20:19:49.125 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom 2024-05-09T20:19:49.146 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/19 kB Progress (1): 19 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/forge/forge-parent/38/forge-parent-38.pom (19 kB at 892 kB/s) 2024-05-09T20:19:49.152 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom 2024-05-09T20:19:49.174 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 363 B Downloaded from central: https://repo.maven.apache.org/maven2/aopalliance/aopalliance/1.0/aopalliance-1.0.pom (363 B at 16 kB/s) 2024-05-09T20:19:49.178 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom 2024-05-09T20:19:49.199 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.1 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava/16.0.1/guava-16.0.1.pom (6.1 kB at 277 kB/s) 2024-05-09T20:19:49.203 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom 2024-05-09T20:19:49.223 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/16.0.1/guava-parent-16.0.1.pom (7.3 kB at 367 kB/s) 2024-05-09T20:19:49.227 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2024-05-09T20:19:49.245 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom (4.8 kB at 284 kB/s) 2024-05-09T20:19:49.250 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom 2024-05-09T20:19:49.269 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 7.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-classworlds/2.5.2/plexus-classworlds-2.5.2.pom (7.3 kB at 406 kB/s) 2024-05-09T20:19:49.276 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom 2024-05-09T20:19:49.298 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/22 kB Progress (1): 22 kB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.pom (22 kB at 984 kB/s) 2024-05-09T20:19:49.303 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom 2024-05-09T20:19:49.328 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/78 kB Progress (1): 33/78 kB Progress (1): 49/78 kB Progress (1): 66/78 kB Progress (1): 78 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/64/commons-parent-64.pom (78 kB at 3.2 MB/s) 2024-05-09T20:19:49.336 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom 2024-05-09T20:19:49.356 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/23 kB Progress (1): 23 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/30/apache-30.pom (23 kB at 1.2 MB/s) 2024-05-09T20:19:49.363 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom 2024-05-09T20:19:49.381 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/junit-bom/5.10.0/junit-bom-5.10.0.pom (5.6 kB at 297 kB/s) 2024-05-09T20:19:49.386 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom 2024-05-09T20:19:49.405 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.pom (4.3 kB at 225 kB/s) 2024-05-09T20:19:49.409 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom 2024-05-09T20:19:49.428 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-languages/1.2.0/plexus-languages-1.2.0.pom (3.2 kB at 168 kB/s) 2024-05-09T20:19:49.671 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom 2024-05-09T20:19:49.671 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/28 kB Progress (1): 28 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/15/plexus-15.pom (28 kB at 1.3 MB/s) 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.4 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.pom (2.4 kB at 132 kB/s) 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/308 kB Progress (1): 33/308 kB Progress (1): 49/308 kB Progress (1): 63/308 kB Progress (1): 80/308 kB Progress (1): 96/308 kB Progress (1): 112/308 kB Progress (1): 129/308 kB Progress (1): 145/308 kB Progress (1): 162/308 kB Progress (1): 178/308 kB Progress (1): 194/308 kB Progress (1): 211/308 kB Progress (1): 227/308 kB Progress (1): 244/308 kB Progress (1): 260/308 kB Progress (1): 276/308 kB Progress (1): 293/308 kB Progress (1): 308 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/maven-surefire-common/3.2.3/maven-surefire-common-3.2.3.jar (308 kB at 7.0 MB/s) 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/171 kB Progress (2): 16/171 kB | 14 kB Progress (3): 16/171 kB | 14 kB | 16/118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-logger-api/3.2.3/surefire-logger-api-3.2.3.jar (14 kB at 647 kB/s) 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 16/171 kB | 16/118 kB | 16/26 kB Progress (3): 33/171 kB | 16/118 kB | 16/26 kB Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar 2024-05-09T20:19:49.672 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 33/171 kB | 16/118 kB | 26 kB Progress (4): 33/171 kB | 16/118 kB | 26 kB | 8.2 kB Progress (4): 33/171 kB | 33/118 kB | 26 kB | 8.2 kB Progress (4): 49/171 kB | 33/118 kB | 26 kB | 8.2 kB Progress (4): 49/171 kB | 49/118 kB | 26 kB | 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-api/3.2.3/surefire-extensions-api-3.2.3.jar (26 kB at 1.1 MB/s) 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-extensions-spi/3.2.3/surefire-extensions-spi-3.2.3.jar (8.2 kB at 372 kB/s) 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 66/171 kB | 49/118 kB Progress (2): 66/171 kB | 66/118 kB Progress (2): 82/171 kB | 66/118 kB Progress (2): 82/171 kB | 82/118 kB Progress (2): 98/171 kB | 82/118 kB Progress (2): 98/171 kB | 98/118 kB Progress (2): 98/171 kB | 115/118 kB Progress (2): 98/171 kB | 118 kB Progress (2): 115/171 kB | 118 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-booter/3.2.3/surefire-booter-3.2.3.jar (118 kB at 4.5 MB/s) 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 131/171 kB Progress (1): 147/171 kB Progress (1): 164/171 kB Progress (1): 171 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-api/3.2.3/surefire-api-3.2.3.jar (171 kB at 4.7 MB/s) 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/146 kB Progress (1): 33/146 kB Progress (1): 49/146 kB Progress (1): 62/146 kB Progress (1): 79/146 kB Progress (1): 95/146 kB Progress (1): 111/146 kB Progress (1): 128/146 kB Progress (1): 144/146 kB Progress (1): 146 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-util/1.0.0.v20140518/aether-util-1.0.0.v20140518.jar (146 kB at 3.7 MB/s) 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/136 kB Progress (2): 16/136 kB | 16/61 kB Progress (2): 32/136 kB | 16/61 kB Progress (2): 32/136 kB | 33/61 kB Progress (2): 49/136 kB | 33/61 kB Progress (2): 49/136 kB | 49/61 kB Progress (2): 65/136 kB | 49/61 kB Progress (2): 65/136 kB | 61 kB Progress (2): 81/136 kB | 61 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-common-artifact-filters/3.1.1/maven-common-artifact-filters-3.1.1.jar (61 kB at 1.5 MB/s) 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 98/136 kB Progress (1): 114/136 kB Progress (1): 130/136 kB Progress (1): 136 kB Progress (2): 136 kB | 16/501 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/eclipse/aether/aether-api/1.0.0.v20140518/aether-api-1.0.0.v20140518.jar (136 kB at 3.2 MB/s) 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 33/501 kB Progress (1): 49/501 kB Progress (1): 66/501 kB Progress (1): 82/501 kB Progress (1): 98/501 kB Progress (1): 115/501 kB Progress (1): 131/501 kB Progress (1): 147/501 kB Progress (1): 164/501 kB Progress (1): 180/501 kB Progress (1): 197/501 kB Progress (1): 213/501 kB Progress (1): 229/501 kB Progress (2): 229/501 kB | 16/58 kB Progress (2): 229/501 kB | 33/58 kB Progress (2): 229/501 kB | 49/58 kB Progress (2): 229/501 kB | 58 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-java/1.2.0/plexus-java-1.2.0.jar (58 kB at 1.1 MB/s) 2024-05-09T20:19:49.673 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 229/501 kB | 16/124 kB Progress (2): 229/501 kB | 33/124 kB Progress (2): 229/501 kB | 49/124 kB Progress (2): 229/501 kB | 65/124 kB Progress (2): 229/501 kB | 81/124 kB Progress (2): 229/501 kB | 97/124 kB Progress (2): 229/501 kB | 114/124 kB Progress (2): 229/501 kB | 124 kB Progress (2): 246/501 kB | 124 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/ow2/asm/asm/9.6/asm-9.6.jar (124 kB at 2.2 MB/s) 2024-05-09T20:19:49.674 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 262/501 kB Progress (1): 279/501 kB Progress (1): 295/501 kB Progress (1): 311/501 kB Progress (1): 328/501 kB Progress (1): 344/501 kB Progress (1): 360/501 kB Progress (1): 377/501 kB Progress (1): 393/501 kB Progress (1): 410/501 kB Progress (2): 410/501 kB | 0/2.4 MB Progress (2): 410/501 kB | 0/2.4 MB Progress (2): 410/501 kB | 0/2.4 MB Progress (2): 426/501 kB | 0/2.4 MB Progress (2): 426/501 kB | 0.1/2.4 MB Progress (2): 426/501 kB | 0.1/2.4 MB Progress (2): 426/501 kB | 0.1/2.4 MB Progress (2): 442/501 kB | 0.1/2.4 MB Progress (2): 442/501 kB | 0.1/2.4 MB Progress (2): 459/501 kB | 0.1/2.4 MB Progress (2): 459/501 kB | 0.1/2.4 MB Progress (2): 475/501 kB | 0.1/2.4 MB Progress (2): 475/501 kB | 0.1/2.4 MB Progress (2): 492/501 kB | 0.1/2.4 MB Progress (2): 492/501 kB | 0.2/2.4 MB Progress (2): 501 kB | 0.2/2.4 MB Progress (2): 501 kB | 0.2/2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/commons-io/commons-io/2.15.0/commons-io-2.15.0.jar (501 kB at 7.4 MB/s) 2024-05-09T20:19:49.674 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.2/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.3/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.4/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.5/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.6/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.7/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.8/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 0.9/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.0/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.1/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.2/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.3/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.4/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.5/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.6/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.7/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.8/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 1.9/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.0/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.1/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.2/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.3/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4/2.4 MB Progress (1): 2.4 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-shared-utils/3.2.3/surefire-shared-utils-3.2.3.jar (2.4 MB at 21 MB/s) 2024-05-09T20:19:49.781 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:49.782 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- jar:3.3.0:jar (default-jar) @ jcksum --- 2024-05-09T20:19:49.787 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom 2024-05-09T20:19:49.805 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.5 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.pom (4.5 kB at 249 kB/s) 2024-05-09T20:19:49.807 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom 2024-05-09T20:19:49.825 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 4.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/maven-shared-components/36/maven-shared-components-36.pom (4.9 kB at 272 kB/s) 2024-05-09T20:19:49.827 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom 2024-05-09T20:19:49.846 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/45 kB Progress (1): 33/45 kB Progress (1): 45 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/36/maven-parent-36.pom (45 kB at 2.4 MB/s) 2024-05-09T20:19:49.849 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom 2024-05-09T20:19:49.867 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/21 kB Progress (1): 21 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/26/apache-26.pom (21 kB at 1.1 MB/s) 2024-05-09T20:19:49.871 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom 2024-05-09T20:19:49.888 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 5.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.3.0/plexus-utils-3.3.0.pom (5.2 kB at 305 kB/s) 2024-05-09T20:19:49.891 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom 2024-05-09T20:19:49.907 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 3.9 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.pom (3.9 kB at 231 kB/s) 2024-05-09T20:19:49.910 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom 2024-05-09T20:19:49.926 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.pom (6.0 kB at 354 kB/s) 2024-05-09T20:19:49.929 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom 2024-05-09T20:19:49.948 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 6.3 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.pom (6.3 kB at 330 kB/s) 2024-05-09T20:19:49.952 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom 2024-05-09T20:19:49.972 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/20 kB Progress (1): 20 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.pom (20 kB at 984 kB/s) 2024-05-09T20:19:49.979 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom 2024-05-09T20:19:50.000 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 15 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.pom (15 kB at 661 kB/s) 2024-05-09T20:19:50.004 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom 2024-05-09T20:19:50.023 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 2.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.pom (2.0 kB at 102 kB/s) 2024-05-09T20:19:50.027 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom 2024-05-09T20:19:50.044 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 8.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.pom (8.2 kB at 456 kB/s) 2024-05-09T20:19:50.048 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom 2024-05-09T20:19:50.066 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/25 kB Progress (1): 25 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus/8/plexus-8.pom (25 kB at 1.3 MB/s) 2024-05-09T20:19:50.078 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar 2024-05-09T20:19:50.102 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/36 kB Progress (1): 33/36 kB Progress (1): 36 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/shared/file-management/3.1.0/file-management-3.1.0.jar (36 kB at 1.5 MB/s) 2024-05-09T20:19:50.102 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar 2024-05-09T20:19:50.103 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar 2024-05-09T20:19:50.103 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar 2024-05-09T20:19:50.103 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar 2024-05-09T20:19:50.104 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar 2024-05-09T20:19:50.121 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/26 kB Progress (2): 16/26 kB | 16/79 kB Progress (2): 26 kB | 16/79 kB Progress (2): 26 kB | 33/79 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-archiver/3.6.0/maven-archiver-3.6.0.jar (26 kB at 1.4 MB/s) 2024-05-09T20:19:50.121 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar 2024-05-09T20:19:50.123 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 49/79 kB Progress (2): 49/79 kB | 16/58 kB Progress (2): 49/79 kB | 33/58 kB Progress (2): 66/79 kB | 33/58 kB Progress (3): 66/79 kB | 33/58 kB | 16/211 kB Progress (3): 66/79 kB | 49/58 kB | 16/211 kB Progress (3): 79 kB | 49/58 kB | 16/211 kB Progress (3): 79 kB | 58 kB | 16/211 kB Progress (3): 79 kB | 58 kB | 33/211 kB Progress (3): 79 kB | 58 kB | 49/211 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/iq80/snappy/snappy/0.4/snappy-0.4.jar (58 kB at 2.9 MB/s) 2024-05-09T20:19:50.123 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-io/3.4.0/plexus-io-3.4.0.jar (79 kB at 3.8 MB/s) 2024-05-09T20:19:50.123 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar 2024-05-09T20:19:50.126 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 66/211 kB Progress (1): 82/211 kB Progress (1): 98/211 kB Progress (1): 115/211 kB Progress (1): 131/211 kB Progress (2): 131/211 kB | 0/1.0 MB Progress (2): 147/211 kB | 0/1.0 MB Progress (2): 164/211 kB | 0/1.0 MB Progress (2): 180/211 kB | 0/1.0 MB Progress (2): 197/211 kB | 0/1.0 MB Progress (2): 211 kB | 0/1.0 MB Progress (2): 211 kB | 0/1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-archiver/4.4.0/plexus-archiver-4.4.0.jar (211 kB at 8.8 MB/s) 2024-05-09T20:19:50.145 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 0/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.1/1.0 MB Progress (1): 0.2/1.0 MB Progress (2): 0.2/1.0 MB | 16/116 kB Progress (2): 0.2/1.0 MB | 32/116 kB Progress (2): 0.2/1.0 MB | 49/116 kB Progress (2): 0.2/1.0 MB | 65/116 kB Progress (2): 0.2/1.0 MB | 81/116 kB Progress (2): 0.2/1.0 MB | 98/116 kB Progress (2): 0.2/1.0 MB | 114/116 kB Progress (2): 0.2/1.0 MB | 116 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar (116 kB at 2.8 MB/s) 2024-05-09T20:19:50.151 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 0.2/1.0 MB Progress (2): 0.2/1.0 MB | 16/267 kB Progress (2): 0.2/1.0 MB | 33/267 kB Progress (2): 0.2/1.0 MB | 49/267 kB Progress (2): 0.2/1.0 MB | 66/267 kB Progress (2): 0.2/1.0 MB | 66/267 kB Progress (2): 0.2/1.0 MB | 82/267 kB Progress (2): 0.2/1.0 MB | 98/267 kB Progress (2): 0.2/1.0 MB | 115/267 kB Progress (2): 0.2/1.0 MB | 131/267 kB Progress (2): 0.2/1.0 MB | 147/267 kB Progress (2): 0.2/1.0 MB | 164/267 kB Progress (2): 0.2/1.0 MB | 164/267 kB Progress (2): 0.2/1.0 MB | 180/267 kB Progress (2): 0.2/1.0 MB | 197/267 kB Progress (2): 0.2/1.0 MB | 213/267 kB Progress (2): 0.2/1.0 MB | 229/267 kB Progress (2): 0.2/1.0 MB | 246/267 kB Progress (2): 0.2/1.0 MB | 246/267 kB Progress (2): 0.2/1.0 MB | 256/267 kB Progress (2): 0.2/1.0 MB | 267 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/codehaus/plexus/plexus-utils/3.4.2/plexus-utils-3.4.2.jar (267 kB at 5.6 MB/s) 2024-05-09T20:19:50.231 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 0.2/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.3/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.4/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.5/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.6/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.7/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.8/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 0.9/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0/1.0 MB Progress (1): 1.0 MB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/commons/commons-compress/1.21/commons-compress-1.21.jar (1.0 MB at 8.0 MB/s) 2024-05-09T20:19:50.415 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Building jar: /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/target/jcksum-1.0.jar 2024-05-09T20:19:50.455 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-09T20:19:50.456 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] BUILD SUCCESS 2024-05-09T20:19:50.456 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-09T20:19:50.456 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Total time: 10.315 s 2024-05-09T20:19:50.457 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Finished at: 2024-05-09T20:19:50Z 2024-05-09T20:19:50.457 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-09T20:19:50.478 INFO:tasks.workunit.client.0.smithi063.stderr:+ ./mvnw test -Dtest=PutObjects 2024-05-09T20:19:50.484 INFO:tasks.workunit.client.0.smithi063.stdout:Warning: JAVA_HOME environment variable is not set. 2024-05-09T20:19:51.722 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Scanning for projects... 2024-05-09T20:19:51.813 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:51.813 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ---------------------------< io.ceph:jcksum >--------------------------- 2024-05-09T20:19:51.813 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Building jcksum 1.0 2024-05-09T20:19:51.813 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] from pom.xml 2024-05-09T20:19:51.813 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --------------------------------[ jar ]--------------------------------- 2024-05-09T20:19:52.070 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:52.070 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- resources:3.3.1:resources (default-resources) @ jcksum --- 2024-05-09T20:19:52.202 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/main/resources 2024-05-09T20:19:52.202 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:52.202 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- compiler:3.11.0:compile (default-compile) @ jcksum --- 2024-05-09T20:19:52.296 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Nothing to compile - all classes are up to date 2024-05-09T20:19:52.296 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:52.296 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- resources:3.3.1:testResources (default-testResources) @ jcksum --- 2024-05-09T20:19:52.299 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] skip non existing resourceDirectory /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/jcksum/src/test/resources 2024-05-09T20:19:52.299 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:52.299 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ jcksum --- 2024-05-09T20:19:52.303 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Nothing to compile - all classes are up to date 2024-05-09T20:19:52.303 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:52.303 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] --- surefire:3.2.3:test (default-test) @ jcksum --- 2024-05-09T20:19:52.494 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider 2024-05-09T20:19:52.736 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom 2024-05-09T20:19:52.942 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/4.7 kB Progress (1): 2.8/4.7 kB Progress (1): 4.1/4.7 kB Progress (1): 4.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.pom (4.7 kB at 22 kB/s) 2024-05-09T20:19:52.950 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom 2024-05-09T20:19:52.974 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/2.6 kB Progress (1): 2.6 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-providers/3.2.3/surefire-providers-3.2.3.pom (2.6 kB at 131 kB/s) 2024-05-09T20:19:52.976 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom 2024-05-09T20:19:53.000 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.pom (2.8 kB at 117 kB/s) 2024-05-09T20:19:53.005 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom 2024-05-09T20:19:53.025 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/3.0 kB Progress (1): 2.8/3.0 kB Progress (1): 3.0 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.pom (3.0 kB at 151 kB/s) 2024-05-09T20:19:53.028 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom 2024-05-09T20:19:53.048 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/3.2 kB Progress (1): 2.8/3.2 kB Progress (1): 3.2 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.pom (3.2 kB at 160 kB/s) 2024-05-09T20:19:53.051 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom 2024-05-09T20:19:53.070 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/1.7 kB Progress (1): 1.7 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.pom (1.7 kB at 89 kB/s) 2024-05-09T20:19:53.073 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom 2024-05-09T20:19:53.092 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/2.8 kB Progress (1): 2.8/2.8 kB Progress (1): 2.8 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.pom (2.8 kB at 149 kB/s) 2024-05-09T20:19:53.098 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar 2024-05-09T20:19:53.125 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/27 kB Progress (1): 2.8/27 kB Progress (1): 4.1/27 kB Progress (1): 5.5/27 kB Progress (1): 6.9/27 kB Progress (1): 8.3/27 kB Progress (1): 9.7/27 kB Progress (1): 11/27 kB Progress (1): 12/27 kB Progress (1): 14/27 kB Progress (1): 15/27 kB Progress (1): 17/27 kB Progress (1): 18/27 kB Progress (1): 19/27 kB Progress (1): 21/27 kB Progress (1): 22/27 kB Progress (1): 23/27 kB Progress (1): 25/27 kB Progress (1): 26/27 kB Progress (1): 27 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire-junit-platform/3.2.3/surefire-junit-platform-3.2.3.jar (27 kB at 1.0 MB/s) 2024-05-09T20:19:53.127 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar 2024-05-09T20:19:53.127 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar 2024-05-09T20:19:53.130 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar 2024-05-09T20:19:53.130 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar 2024-05-09T20:19:53.134 INFO:tasks.workunit.client.0.smithi063.stdout:Downloading from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar 2024-05-09T20:19:53.155 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 16/18 kB Progress (1): 18 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/common-java5/3.2.3/common-java5-3.2.3.jar (18 kB at 585 kB/s) 2024-05-09T20:19:53.199 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (1): 1.4/7.7 kB Progress (1): 2.8/7.7 kB Progress (1): 4.1/7.7 kB Progress (2): 4.1/7.7 kB | 1.4/103 kB Progress (3): 4.1/7.7 kB | 1.4/103 kB | 1.4/169 kB Progress (4): 4.1/7.7 kB | 1.4/103 kB | 1.4/169 kB | 1.4/189 kB Progress (4): 4.1/7.7 kB | 1.4/103 kB | 2.8/169 kB | 1.4/189 kB Progress (4): 5.5/7.7 kB | 1.4/103 kB | 2.8/169 kB | 1.4/189 kB Progress (4): 5.5/7.7 kB | 1.4/103 kB | 4.1/169 kB | 1.4/189 kB Progress (4): 5.5/7.7 kB | 1.4/103 kB | 4.1/169 kB | 2.8/189 kB Progress (4): 5.5/7.7 kB | 2.8/103 kB | 4.1/169 kB | 2.8/189 kB Progress (4): 5.5/7.7 kB | 2.8/103 kB | 4.1/169 kB | 4.1/189 kB Progress (4): 5.5/7.7 kB | 2.8/103 kB | 5.5/169 kB | 4.1/189 kB Progress (4): 6.9/7.7 kB | 2.8/103 kB | 5.5/169 kB | 4.1/189 kB Progress (4): 6.9/7.7 kB | 2.8/103 kB | 6.9/169 kB | 4.1/189 kB Progress (4): 6.9/7.7 kB | 4.1/103 kB | 6.9/169 kB | 4.1/189 kB Progress (4): 6.9/7.7 kB | 4.1/103 kB | 6.9/169 kB | 5.5/189 kB Progress (4): 6.9/7.7 kB | 4.1/103 kB | 8.3/169 kB | 5.5/189 kB Progress (4): 7.7 kB | 4.1/103 kB | 8.3/169 kB | 5.5/189 kB Progress (4): 7.7 kB | 4.1/103 kB | 9.7/169 kB | 5.5/189 kB Progress (4): 7.7 kB | 4.1/103 kB | 9.7/169 kB | 6.9/189 kB Progress (4): 7.7 kB | 4.1/103 kB | 11/169 kB | 6.9/189 kB Progress (4): 7.7 kB | 5.5/103 kB | 11/169 kB | 6.9/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/opentest4j/opentest4j/1.2.0/opentest4j-1.2.0.jar (7.7 kB at 113 kB/s) 2024-05-09T20:19:53.235 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (3): 5.5/103 kB | 11/169 kB | 8.3/189 kB Progress (3): 6.9/103 kB | 11/169 kB | 8.3/189 kB Progress (3): 6.9/103 kB | 12/169 kB | 8.3/189 kB Progress (3): 8.3/103 kB | 12/169 kB | 8.3/189 kB Progress (3): 8.3/103 kB | 12/169 kB | 9.7/189 kB Progress (3): 8.3/103 kB | 14/169 kB | 9.7/189 kB Progress (3): 8.3/103 kB | 14/169 kB | 11/189 kB Progress (3): 9.7/103 kB | 14/169 kB | 11/189 kB Progress (3): 9.7/103 kB | 14/169 kB | 12/189 kB Progress (3): 9.7/103 kB | 15/169 kB | 12/189 kB Progress (3): 9.7/103 kB | 15/169 kB | 14/189 kB Progress (3): 11/103 kB | 15/169 kB | 14/189 kB Progress (3): 11/103 kB | 15/169 kB | 15/189 kB Progress (3): 11/103 kB | 16/169 kB | 15/189 kB Progress (3): 12/103 kB | 16/169 kB | 15/189 kB Progress (3): 12/103 kB | 16/169 kB | 16/189 kB Progress (3): 14/103 kB | 16/169 kB | 16/189 kB Progress (3): 15/103 kB | 16/169 kB | 16/189 kB Progress (3): 16/103 kB | 16/169 kB | 16/189 kB Progress (3): 16/103 kB | 16/169 kB | 32/189 kB Progress (3): 32/103 kB | 16/169 kB | 32/189 kB Progress (3): 32/103 kB | 32/169 kB | 32/189 kB Progress (3): 32/103 kB | 32/169 kB | 49/189 kB Progress (3): 49/103 kB | 32/169 kB | 49/189 kB Progress (3): 49/103 kB | 32/169 kB | 65/189 kB Progress (3): 49/103 kB | 49/169 kB | 65/189 kB Progress (3): 65/103 kB | 49/169 kB | 65/189 kB Progress (3): 65/103 kB | 65/169 kB | 65/189 kB Progress (3): 65/103 kB | 81/169 kB | 65/189 kB Progress (3): 65/103 kB | 81/169 kB | 81/189 kB Progress (3): 81/103 kB | 81/169 kB | 81/189 kB Progress (3): 81/103 kB | 98/169 kB | 81/189 kB Progress (3): 81/103 kB | 98/169 kB | 98/189 kB Progress (3): 98/103 kB | 98/169 kB | 98/189 kB Progress (3): 98/103 kB | 98/169 kB | 114/189 kB Progress (3): 103 kB | 98/169 kB | 114/189 kB Progress (3): 103 kB | 98/169 kB | 130/189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-commons/1.9.3/junit-platform-commons-1.9.3.jar (103 kB at 981 kB/s) 2024-05-09T20:19:53.242 INFO:tasks.workunit.client.0.smithi063.stdout:Progress (2): 114/169 kB | 130/189 kB Progress (2): 130/169 kB | 130/189 kB Progress (2): 130/169 kB | 147/189 kB Progress (2): 147/169 kB | 147/189 kB Progress (2): 147/169 kB | 163/189 kB Progress (2): 163/169 kB | 163/189 kB Progress (2): 169 kB | 163/189 kB Progress (2): 169 kB | 180/189 kB Progress (2): 169 kB | 189 kB Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-launcher/1.9.3/junit-platform-launcher-1.9.3.jar (169 kB at 1.5 MB/s) 2024-05-09T20:19:53.243 INFO:tasks.workunit.client.0.smithi063.stdout:Downloaded from central: https://repo.maven.apache.org/maven2/org/junit/platform/junit-platform-engine/1.9.3/junit-platform-engine-1.9.3.jar (189 kB at 1.6 MB/s) 2024-05-09T20:19:53.263 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:19:53.263 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ------------------------------------------------------- 2024-05-09T20:19:53.264 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] T E S T S 2024-05-09T20:19:53.264 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ------------------------------------------------------- 2024-05-09T20:19:53.942 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Running io.ceph.jcksum.PutObjects 2024-05-09T20:19:53.979 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjects.java: starting test run: 2024-05-09T20:19:54.077 INFO:tasks.workunit.client.0.smithi063.stdout: AccessKey=lNCnR47C2g+ZidCWBAUuwfSAA7Q= 2024-05-09T20:19:54.077 INFO:tasks.workunit.client.0.smithi063.stdout: SecretKey=tYuA2Y+Uu1ow2l9Xe59tWKVml3gMuVfyhUjjJwfwEI0vFFONIcqf4g== 2024-05-09T20:19:54.077 INFO:tasks.workunit.client.0.smithi063.stdout: EndpointUrl=http://smithi063.front.sepia.ceph.com:80 2024-05-09T20:19:54.078 INFO:tasks.workunit.client.0.smithi063.stderr:[main] WARN software.amazon.awssdk.http.apache.ApacheHttpClient - SSL Certificate verification is disabled. This is not a safe setting and should only be used for testing. 2024-05-09T20:19:54.277 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Environment Variables Read 2024-05-09T20:19:54.578 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: S3 Client Initialized 2024-05-09T20:19:54.578 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Generating File 2024-05-09T20:19:54.578 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: File Generated 2024-05-09T20:19:54.578 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Generating File 2024-05-09T20:19:54.578 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: File Generated 2024-05-09T20:19:54.578 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Generating File 2024-05-09T20:19:54.679 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: File Generated 2024-05-09T20:19:54.679 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Generating File 2024-05-09T20:19:54.679 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: File Generated 2024-05-09T20:19:54.679 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Generating File 2024-05-09T20:19:54.679 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: File Generated 2024-05-09T20:19:54.679 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Generating File 2024-05-09T20:19:54.679 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: File Generated 2024-05-09T20:19:54.679 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Generated Big Files 2024-05-09T20:19:55.079 INFO:tasks.workunit.client.0.smithi063.stdout:HeadBucketResponse() 2024-05-09T20:19:55.079 INFO:tasks.workunit.client.0.smithi063.stdout:sheik is ready 2024-05-09T20:19:55.079 INFO:tasks.workunit.client.0.smithi063.stdout:DEBUG: Test Bucket Created 2024-05-09T20:19:55.179 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-0b 2024-05-09T20:19:55.480 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-8b 2024-05-09T20:19:55.781 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-200b 2024-05-09T20:19:56.081 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-21983b 2024-05-09T20:19:56.382 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-5519b 2024-05-09T20:19:56.583 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-204329b 2024-05-09T20:19:56.984 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-256k 2024-05-09T20:19:57.284 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-1m 2024-05-09T20:19:57.986 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileNoCksum called with file-1038757b 2024-05-09T20:19:58.688 INFO:tasks.workunit.client.0.smithi063.stdout:mpuObjectFromFileNoCksum called with file-5m 2024-05-09T20:20:03.699 INFO:tasks.workunit.client.0.smithi063.stdout:mpuObjectFromFileNoCksum called with file-10m 2024-05-09T20:20:10.413 INFO:tasks.workunit.client.0.smithi063.stdout:mpuObjectFromFileNoCksum called with file-100m 2024-05-09T20:20:51.967 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-0b 2024-05-09T20:20:51.968 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-8b 2024-05-09T20:20:51.970 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-200b 2024-05-09T20:20:51.972 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-21983b 2024-05-09T20:20:51.974 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-5519b 2024-05-09T20:20:51.975 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-204329b 2024-05-09T20:20:51.977 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-256k 2024-05-09T20:20:51.979 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-1m 2024-05-09T20:20:51.980 INFO:tasks.workunit.client.0.smithi063.stdout:arg: file-1038757b 2024-05-09T20:20:51.984 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-0b 2024-05-09T20:20:52.282 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:52.285 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-8b 2024-05-09T20:20:52.485 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:52.586 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-200b 2024-05-09T20:20:52.786 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:52.886 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-21983b 2024-05-09T20:20:52.986 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:53.187 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-5519b 2024-05-09T20:20:53.287 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:53.387 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-204329b 2024-05-09T20:20:53.687 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:53.988 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-256k 2024-05-09T20:20:54.188 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:54.288 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-1m 2024-05-09T20:20:54.789 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:55.090 INFO:tasks.workunit.client.0.smithi063.stdout:putObjectFromFileCksum called with file-1038757b 2024-05-09T20:20:55.490 INFO:tasks.workunit.client.0.smithi063.stdout:PutObjectResponse 2024-05-09T20:20:55.690 INFO:tasks.workunit.client.0.smithi063.stdout:mpuObjectFromFileCksum called with file-5m 2024-05-09T20:20:59.296 INFO:tasks.workunit.client.0.smithi063.stdout:mpuObjectFromFileCksum called with file-10m 2024-05-09T20:21:03.801 INFO:tasks.workunit.client.0.smithi063.stdout:mpuObjectFromFileCksum called with file-100m 2024-05-09T20:22:05.370 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 131.4 s -- in io.ceph.jcksum.PutObjects 2024-05-09T20:22:05.400 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:22:05.401 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Results: 2024-05-09T20:22:05.401 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:22:05.401 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Tests run: 33, Failures: 0, Errors: 0, Skipped: 0 2024-05-09T20:22:05.401 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] 2024-05-09T20:22:05.403 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-09T20:22:05.403 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] BUILD SUCCESS 2024-05-09T20:22:05.403 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-09T20:22:05.404 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Total time: 02:13 min 2024-05-09T20:22:05.405 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] Finished at: 2024-05-09T20:22:05Z 2024-05-09T20:22:05.405 INFO:tasks.workunit.client.0.smithi063.stdout:[INFO] ------------------------------------------------------------------------ 2024-05-09T20:22:05.425 INFO:tasks.workunit.client.0.smithi063.stderr:+ exit 2024-05-09T20:22:05.425 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:22:05.425 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:22:05.476 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'cls/test_cls_rgw_gc.sh', 'cls/test_cls_rgw_stats.sh', 'cls/test_cls_cmpomap.sh', 'cls/test_cls_2pc_queue.sh', 'cls/test_cls_user.sh', 'rgw/test_rgw_gc_log.sh', 'rgw/test_rgw_obj.sh', 'rgw/test_librgw_file.sh', 'rgw/test_awssdkv4_sig.sh'] on client.0... 2024-05-09T20:22:05.476 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T20:22:05.869 DEBUG:teuthology.parallel:result is None 2024-05-09T20:22:05.869 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:22:05.883 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:22:05.883 DEBUG:teuthology.orchestra.run.smithi063:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:22:05.930 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:22:05.930 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T20:22:05.940 INFO:tasks.workunit:Pulling workunits from ref 86872c6b9201173c508ab909aaa65bdb12239dd6 2024-05-09T20:22:05.940 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T20:22:05.940 DEBUG:teuthology.orchestra.run.smithi063:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:22:05.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:22:05.979 INFO:teuthology.orchestra.run.smithi063.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-09T20:22:05.979 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:22:06.026 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-09T20:22:06.027 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-09T20:22:06.075 INFO:tasks.workunit:timeout=3h 2024-05-09T20:22:06.075 INFO:tasks.workunit:cleanup=True 2024-05-09T20:22:06.075 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 86872c6b9201173c508ab909aaa65bdb12239dd6 2024-05-09T20:22:06.124 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T20:22:46.498 INFO:tasks.workunit.client.0.smithi063.stderr:Note: switching to '86872c6b9201173c508ab909aaa65bdb12239dd6'. 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr: git switch -c 2024-05-09T20:22:46.499 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:22:46.500 INFO:tasks.workunit.client.0.smithi063.stderr:Or undo this operation with: 2024-05-09T20:22:46.500 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:22:46.500 INFO:tasks.workunit.client.0.smithi063.stderr: git switch - 2024-05-09T20:22:46.500 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:22:46.500 INFO:tasks.workunit.client.0.smithi063.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T20:22:46.500 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:22:46.500 INFO:tasks.workunit.client.0.smithi063.stderr:HEAD is now at 86872c6b920 d4n: Add more in-depth config descriptions and improve naming 2024-05-09T20:22:46.504 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T20:22:46.552 INFO:tasks.workunit.client.0.smithi063.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T20:22:46.554 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T20:22:46.554 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T20:22:46.608 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T20:22:46.663 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T20:22:46.712 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T20:22:46.713 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T20:22:46.713 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T20:22:46.764 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T20:22:46.768 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:22:46.768 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T20:22:46.814 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_s3_mp_reupload.sh on client.0... 2024-05-09T20:22:46.815 INFO:tasks.workunit:Running workunit rgw/test_rgw_s3_mp_reupload.sh... 2024-05-09T20:22:46.816 DEBUG:teuthology.orchestra.run.smithi063:workunit test rgw/test_rgw_s3_mp_reupload.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_s3_mp_reupload.sh 2024-05-09T20:22:46.870 INFO:tasks.workunit.client.0.smithi063.stdout:INFO: fully qualified domain name: smithi063.front.sepia.ceph.com 2024-05-09T20:22:46.870 INFO:tasks.workunit.client.0.smithi063.stdout:== TESTING *ABORTING* MULTIPART UPLOAD WITH RE-UPLOADS == 2024-05-09T20:22:46.870 INFO:tasks.workunit.client.0.smithi063.stdout:== TESTING VERSIONED BUCKET == 2024-05-09T20:22:50.243 INFO:tasks.workunit.client.0.smithi063.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-05-09T20:22:50.548 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting pip 2024-05-09T20:22:50.555 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-05-09T20:22:50.594 INFO:tasks.workunit.client.0.smithi063.stdout:Installing collected packages: pip 2024-05-09T20:22:50.594 INFO:tasks.workunit.client.0.smithi063.stdout: Attempting uninstall: pip 2024-05-09T20:22:50.595 INFO:tasks.workunit.client.0.smithi063.stdout: Found existing installation: pip 22.0.2 2024-05-09T20:22:50.870 INFO:tasks.workunit.client.0.smithi063.stdout: Uninstalling pip-22.0.2: 2024-05-09T20:22:50.877 INFO:tasks.workunit.client.0.smithi063.stdout: Successfully uninstalled pip-22.0.2 2024-05-09T20:22:51.838 INFO:tasks.workunit.client.0.smithi063.stdout:Successfully installed pip-24.0 2024-05-09T20:22:52.988 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting boto3 2024-05-09T20:22:52.989 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached boto3-1.34.101-py3-none-any.whl.metadata (6.6 kB) 2024-05-09T20:22:53.580 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting botocore<1.35.0,>=1.34.101 (from boto3) 2024-05-09T20:22:53.581 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached botocore-1.34.101-py3-none-any.whl.metadata (5.7 kB) 2024-05-09T20:22:53.657 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-05-09T20:22:53.658 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-05-09T20:22:53.741 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-05-09T20:22:53.742 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-05-09T20:22:53.842 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:22:53.844 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-05-09T20:22:54.007 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:22:54.009 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-05-09T20:22:54.229 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:22:54.231 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-05-09T20:22:54.246 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached boto3-1.34.101-py3-none-any.whl (139 kB) 2024-05-09T20:22:54.247 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached botocore-1.34.101-py3-none-any.whl (12.2 MB) 2024-05-09T20:22:54.275 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-05-09T20:22:54.276 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-05-09T20:22:54.278 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-05-09T20:22:54.279 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-05-09T20:22:54.281 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-09T20:22:54.384 INFO:tasks.workunit.client.0.smithi063.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-05-09T20:22:55.494 INFO:tasks.workunit.client.0.smithi063.stdout:Successfully installed boto3-1.34.101 botocore-1.34.101 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-05-09T20:22:56.263 INFO:tasks.workunit.client.0.smithi063.stdout:INFO: creating user 2024-05-09T20:22:56.636 INFO:tasks.workunit.client.0.smithi063.stdout:{ 2024-05-09T20:22:56.637 INFO:tasks.workunit.client.0.smithi063.stdout: "user_id": "testid", 2024-05-09T20:22:56.637 INFO:tasks.workunit.client.0.smithi063.stdout: "display_name": "M. Tester", 2024-05-09T20:22:56.637 INFO:tasks.workunit.client.0.smithi063.stdout: "email": "tester@ceph.com", 2024-05-09T20:22:56.637 INFO:tasks.workunit.client.0.smithi063.stdout: "suspended": 0, 2024-05-09T20:22:56.637 INFO:tasks.workunit.client.0.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:22:56.637 INFO:tasks.workunit.client.0.smithi063.stdout: "subusers": [], 2024-05-09T20:22:56.638 INFO:tasks.workunit.client.0.smithi063.stdout: "keys": [ 2024-05-09T20:22:56.638 INFO:tasks.workunit.client.0.smithi063.stdout: { 2024-05-09T20:22:56.638 INFO:tasks.workunit.client.0.smithi063.stdout: "user": "testid", 2024-05-09T20:22:56.638 INFO:tasks.workunit.client.0.smithi063.stdout: "access_key": "0555b35654ad1656d804", 2024-05-09T20:22:56.638 INFO:tasks.workunit.client.0.smithi063.stdout: "secret_key": "h7GhxuBLTrlhVUyxSPUKUV8r/2EI4ngqJxD7iBdBYLhwluN30JaT3Q==", 2024-05-09T20:22:56.638 INFO:tasks.workunit.client.0.smithi063.stdout: "active": true, 2024-05-09T20:22:56.639 INFO:tasks.workunit.client.0.smithi063.stdout: "create_date": "2024-05-09T20:22:56.631946Z" 2024-05-09T20:22:56.639 INFO:tasks.workunit.client.0.smithi063.stdout: } 2024-05-09T20:22:56.639 INFO:tasks.workunit.client.0.smithi063.stdout: ], 2024-05-09T20:22:56.639 INFO:tasks.workunit.client.0.smithi063.stdout: "swift_keys": [], 2024-05-09T20:22:56.639 INFO:tasks.workunit.client.0.smithi063.stdout: "caps": [], 2024-05-09T20:22:56.639 INFO:tasks.workunit.client.0.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "default_placement": "", 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "placement_tags": [], 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "bucket_quota": { 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "enabled": false, 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size": -1, 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "max_objects": -1 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "user_quota": { 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "enabled": false, 2024-05-09T20:22:56.640 INFO:tasks.workunit.client.0.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size": -1, 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "max_objects": -1 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: }, 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "type": "rgw", 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "account_id": "", 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "path": "/", 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "create_date": "2024-05-09T20:22:56.631922Z", 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "tags": [], 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: "group_ids": [] 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout:} 2024-05-09T20:22:56.641 INFO:tasks.workunit.client.0.smithi063.stdout: 2024-05-09T20:22:57.135 INFO:tasks.workunit.client.0.smithi063.stderr:5+0 records in 2024-05-09T20:22:57.135 INFO:tasks.workunit.client.0.smithi063.stderr:5+0 records out 2024-05-09T20:22:57.135 INFO:tasks.workunit.client.0.smithi063.stderr:5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.022215 s, 236 MB/s 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:bucket nams is reupload-bkt-131891 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:endpoint is http://smithi063.front.sepia.ceph.com:80 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:start UploadId=2~F6GE8rZzfKhgDO5J65s0608-VMcOBKJ 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:phase 1 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:phase 1 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 2} 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.967 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.968 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.968 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.968 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.968 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.968 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.968 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.968 INFO:tasks.workunit.client.0.smithi063.stdout:phase 2 uploaded part {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1} 2024-05-09T20:23:12.968 INFO:tasks.workunit.client.0.smithi063.stdout:aborting multipart upload, parts=[{'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 1}, {'ETag': '"5851a3ed3e4651d14ba1701a994f5857"', 'PartNumber': 2}] 2024-05-09T20:23:14.433 INFO:tasks.workunit.client.0.smithi063.stdout:radosgw-admin bucket radoslist: 2024-05-09T20:23:14.517 INFO:tasks.workunit.client.0.smithi063.stdout:rados ls: 2024-05-09T20:23:14.539 INFO:tasks.workunit.client.0.smithi063.stdout:== PASSED == 2024-05-09T20:23:14.539 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:23:14.539 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:23:14.592 INFO:tasks.workunit:Stopping ['rgw/test_rgw_s3_mp_reupload.sh'] on client.0... 2024-05-09T20:23:14.592 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T20:23:15.000 DEBUG:teuthology.parallel:result is None 2024-05-09T20:23:15.001 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:23:15.012 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:23:15.012 DEBUG:teuthology.orchestra.run.smithi063:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:23:15.061 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:23:15.061 INFO:teuthology.run_tasks:Running task ragweed... 2024-05-09T20:23:15.078 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'default-branch': 'ceph-master', 'rgw_server': 'client.0', 'stages': 'prepare,check'}} 2024-05-09T20:23:15.078 INFO:tasks.ragweed:Downloading ragweed... 2024-05-09T20:23:15.078 INFO:tasks.ragweed:Using branch 'wip-rgw-d4n-long-descriptions' for ragweed 2024-05-09T20:23:15.079 DEBUG:teuthology.orchestra.run.smithi063:> git clone -b wip-rgw-d4n-long-descriptions https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-05-09T20:23:15.110 INFO:teuthology.orchestra.run.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-05-09T20:23:15.160 INFO:teuthology.orchestra.run.smithi063.stderr:fatal: Remote branch wip-rgw-d4n-long-descriptions not found in upstream origin 2024-05-09T20:23:15.161 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-05-09T20:23:15.162 INFO:tasks.ragweed:Using branch 'ceph-master' for ragweed 2024-05-09T20:23:15.162 DEBUG:teuthology.orchestra.run.smithi063:> git clone -b ceph-master https://git.ceph.com/ragweed.git /home/ubuntu/cephtest/ragweed.client.0 2024-05-09T20:23:15.211 INFO:teuthology.orchestra.run.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/ragweed.client.0'... 2024-05-09T20:23:15.375 INFO:tasks.ragweed:Creating rgw users... 2024-05-09T20:23:15.375 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2024-05-09T20:23:15.376 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key MAGDZXYRKRYDQIPQWLYT --secret poQ5idZAZXEWwk/kKAwVbTfUVsOqKfwzbf4mSqDoSnWFn0xvyD17Ag== --email ragweed.client.0+test@test.test --system false 2024-05-09T20:23:15.466 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:23:15.466 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:23:15.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.515+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 realm 2024-05-09T20:23:15.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:23:15.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:23:15.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.519+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.523+0000 7f0a0a058980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:23:15.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.523+0000 7f0a0a058980 20 searching for the correct realm 2024-05-09T20:23:15.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.523+0000 7f0a0836d640 0 -- 172.21.15.63:0/966849868 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f09fc01b040 msgr2=0x7f09fc08da00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:23:15.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.735+0000 7f0a0736b640 0 -- 172.21.15.63:0/966849868 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f09f805e830 msgr2=0x7f09f806aaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:23:15.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.751+0000 7f0a0a058980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:23:15.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.751+0000 7f0a0a058980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:23:15.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.751+0000 7f0a0a058980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:23:15.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.751+0000 7f0a0a058980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:23:15.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.751+0000 7f0a0a058980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:23:15.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.751+0000 7f0a0a058980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:23:15.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.751+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:23:15.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:23:15.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 zone default found 2024-05-09T20:23:15.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 4 Realm: () 2024-05-09T20:23:15.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:23:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:23:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:23:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 zonegroup default 2024-05-09T20:23:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.755+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.759+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.759+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.759+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.759+0000 7f0a0a058980 20 started sync module instance, tier type = 2024-05-09T20:23:15.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.759+0000 7f0a0a058980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:23:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.779+0000 7f0a0a058980 20 add_watcher() i=0 2024-05-09T20:23:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.779+0000 7f0a0a058980 20 add_watcher() i=1 2024-05-09T20:23:15.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.779+0000 7f0a0a058980 20 add_watcher() i=2 2024-05-09T20:23:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.779+0000 7f0a0a058980 20 add_watcher() i=3 2024-05-09T20:23:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.779+0000 7f0a0a058980 20 add_watcher() i=4 2024-05-09T20:23:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.779+0000 7f0a0a058980 20 add_watcher() i=5 2024-05-09T20:23:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.779+0000 7f0a0a058980 20 add_watcher() i=6 2024-05-09T20:23:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.783+0000 7f0a0a058980 20 add_watcher() i=7 2024-05-09T20:23:15.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.783+0000 7f0a0a058980 2 all 8 watchers are set, enabling cache 2024-05-09T20:23:15.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f08be7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:23:15.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:23:15.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:23:15.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 5 note: GC not initialized 2024-05-09T20:23:15.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f08bd7e2640 20 reqs_thread_entry: start 2024-05-09T20:23:15.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 20 init_complete bucket index max shards: 11 2024-05-09T20:23:15.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 20 Filter name: none 2024-05-09T20:23:15.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f08b6ffd640 20 reqs_thread_entry: start 2024-05-09T20:23:15.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-05-09T20:23:15.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2024-05-09T20:23:15.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-05-09T20:23:15.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2024-05-09T20:23:15.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.787+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2024-05-09T20:23:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-05-09T20:23:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 10 cache get: name=default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT : miss 2024-05-09T20:23:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 10 cache put: name=default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT info.flags=0x0 2024-05-09T20:23:15.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 10 adding default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT to cache LRU end 2024-05-09T20:23:15.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 10 cache get: name=default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT : hit (negative entry) 2024-05-09T20:23:15.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.791+0000 7f0a0a058980 10 cache get: name=default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT : hit (negative entry) 2024-05-09T20:23:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.795+0000 7f0a0a058980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-05-09T20:23:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.795+0000 7f0a0a058980 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-05-09T20:23:15.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.795+0000 7f0a0a058980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-05-09T20:23:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.795+0000 7f08e6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693204 cookie 94044980052736 notifier 5573 bl.length()=675 2024-05-09T20:23:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.795+0000 7f08e6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2024-05-09T20:23:15.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.795+0000 7f08e6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-05-09T20:23:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.803+0000 7f0a0a058980 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-05-09T20:23:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.803+0000 7f0a0a058980 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-05-09T20:23:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.803+0000 7f0a0a058980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-05-09T20:23:15.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.803+0000 7f08e67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693240 cookie 94044980044768 notifier 5573 bl.length()=202 2024-05-09T20:23:15.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.803+0000 7f08e67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2024-05-09T20:23:15.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.803+0000 7f08e67fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2024-05-09T20:23:15.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.807+0000 7f0a0a058980 10 cache put: name=default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT info.flags=0x7 2024-05-09T20:23:15.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.807+0000 7f0a0a058980 10 moving default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT to cache LRU end 2024-05-09T20:23:15.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.807+0000 7f0a0a058980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:MAGDZXYRKRYDQIPQWLYT, ofs0, ns] 2024-05-09T20:23:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.807+0000 7f08e6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693198 cookie 94044980048800 notifier 5573 bl.length()=190 2024-05-09T20:23:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.807+0000 7f08e6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT info.flags=0x7 2024-05-09T20:23:15.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.807+0000 7f08e6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MAGDZXYRKRYDQIPQWLYT to cache LRU end 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "ragweed.client.0", 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr. ragweed.client.0", 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "ragweed.client.0+test@test.test", 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "ragweed.client.0", 2024-05-09T20:23:15.803 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "MAGDZXYRKRYDQIPQWLYT", 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "poQ5idZAZXEWwk/kKAwVbTfUVsOqKfwzbf4mSqDoSnWFn0xvyD17Ag==", 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:23:15.795724Z" 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:23:15.804 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "rgw", 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "", 2024-05-09T20:23:15.805 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:23:15.806 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:23:15.795679Z", 2024-05-09T20:23:15.806 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:23:15.806 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:23:15.806 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:23:15.806 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:23:15.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.811+0000 7f08be7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:23:15.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.819+0000 7f0a0a058980 20 remove_watcher() i=0 2024-05-09T20:23:15.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.819+0000 7f0a0a058980 2 removed watcher, disabling cache 2024-05-09T20:23:15.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.819+0000 7f0a0a058980 20 remove_watcher() i=1 2024-05-09T20:23:15.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.823+0000 7f0a0a058980 20 remove_watcher() i=2 2024-05-09T20:23:15.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.823+0000 7f0a0a058980 20 remove_watcher() i=3 2024-05-09T20:23:15.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.827+0000 7f0a0a058980 20 remove_watcher() i=4 2024-05-09T20:23:15.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.827+0000 7f0a0a058980 20 remove_watcher() i=5 2024-05-09T20:23:15.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.831+0000 7f0a0a058980 20 remove_watcher() i=6 2024-05-09T20:23:15.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.831+0000 7f0a0a058980 20 remove_watcher() i=7 2024-05-09T20:23:15.841 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2024-05-09T20:23:15.841 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key WUUFATBQMBNHMZLDLEIV --secret VCCRmH18IqT9dUxFF8+8XxumaEs5d1AAQmOSIQTLiAE8bMSyY3dD5A== --email sysuser.client.0+test@test.test --system true 2024-05-09T20:23:15.888 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:23:15.889 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:23:15.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.935+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 realm 2024-05-09T20:23:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:23:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:15.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:23:15.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:23:15.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.939+0000 7f00f6f62980 20 searching for the correct realm 2024-05-09T20:23:15.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:15.943+0000 7f00f4a76640 0 -- 172.21.15.63:0/2404792120 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f00ec02f060 msgr2=0x7f00ec05ab40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:23:16.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.159+0000 7f00f4a76640 0 -- 172.21.15.63:0/2404792120 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f00ec02f060 msgr2=0x7f00ec05ab40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:23:16.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:23:16.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:23:16.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:23:16.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:23:16.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:23:16.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:23:16.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:16.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:23:16.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:23:16.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 zone default found 2024-05-09T20:23:16.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 4 Realm: () 2024-05-09T20:23:16.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:23:16.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:23:16.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:23:16.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 zonegroup default 2024-05-09T20:23:16.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:16.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.171+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.175+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:16.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.175+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.175+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:16.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.175+0000 7f00f6f62980 20 started sync module instance, tier type = 2024-05-09T20:23:16.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.175+0000 7f00f6f62980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:23:16.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 20 add_watcher() i=0 2024-05-09T20:23:16.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 20 add_watcher() i=1 2024-05-09T20:23:16.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 20 add_watcher() i=2 2024-05-09T20:23:16.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 20 add_watcher() i=3 2024-05-09T20:23:16.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 20 add_watcher() i=4 2024-05-09T20:23:16.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 20 add_watcher() i=5 2024-05-09T20:23:16.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 20 add_watcher() i=6 2024-05-09T20:23:16.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 20 add_watcher() i=7 2024-05-09T20:23:16.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.199+0000 7f00f6f62980 2 all 8 watchers are set, enabling cache 2024-05-09T20:23:16.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7effaafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:23:16.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7f00f6f62980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:23:16.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7f00f6f62980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:23:16.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7f00f6f62980 5 note: GC not initialized 2024-05-09T20:23:16.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7effa9fe3640 20 reqs_thread_entry: start 2024-05-09T20:23:16.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7f00f6f62980 20 init_complete bucket index max shards: 11 2024-05-09T20:23:16.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7f00f6f62980 20 Filter name: none 2024-05-09T20:23:16.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7eff1b7fe640 20 reqs_thread_entry: start 2024-05-09T20:23:16.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7f00f6f62980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-05-09T20:23:16.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.203+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:16.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2024-05-09T20:23:16.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-05-09T20:23:16.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2024-05-09T20:23:16.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:16.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2024-05-09T20:23:16.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-05-09T20:23:16.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 cache get: name=default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV : miss 2024-05-09T20:23:16.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 20 rados->read ofs=0 len=0 2024-05-09T20:23:16.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:23:16.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 cache put: name=default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV info.flags=0x0 2024-05-09T20:23:16.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 adding default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV to cache LRU end 2024-05-09T20:23:16.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 cache get: name=default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV : hit (negative entry) 2024-05-09T20:23:16.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.207+0000 7f00f6f62980 10 cache get: name=default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV : hit (negative entry) 2024-05-09T20:23:16.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.211+0000 7f00f6f62980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-05-09T20:23:16.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.211+0000 7f00f6f62980 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-05-09T20:23:16.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.211+0000 7f00f6f62980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-05-09T20:23:16.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.211+0000 7effcbfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693241 cookie 94342102759456 notifier 5901 bl.length()=675 2024-05-09T20:23:16.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.211+0000 7effcbfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2024-05-09T20:23:16.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.211+0000 7effcbfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-05-09T20:23:16.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.215+0000 7f00f6f62980 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-05-09T20:23:16.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.215+0000 7f00f6f62980 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-05-09T20:23:16.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.215+0000 7f00f6f62980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-05-09T20:23:16.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.215+0000 7effcb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693205 cookie 94342102765744 notifier 5901 bl.length()=202 2024-05-09T20:23:16.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.215+0000 7effcb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2024-05-09T20:23:16.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.215+0000 7effcb7fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2024-05-09T20:23:16.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.219+0000 7f00f6f62980 10 cache put: name=default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV info.flags=0x7 2024-05-09T20:23:16.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.219+0000 7f00f6f62980 10 moving default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV to cache LRU end 2024-05-09T20:23:16.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.219+0000 7f00f6f62980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.keys:WUUFATBQMBNHMZLDLEIV, ofs0, ns] 2024-05-09T20:23:16.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.219+0000 7effcb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1090921693196 cookie 94342102770672 notifier 5901 bl.length()=190 2024-05-09T20:23:16.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.219+0000 7effcb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV info.flags=0x7 2024-05-09T20:23:16.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.219+0000 7effcb7fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+WUUFATBQMBNHMZLDLEIV to cache LRU end 2024-05-09T20:23:16.215 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:23:16.215 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "sysuser.client.0", 2024-05-09T20:23:16.215 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr. sysuser.client.0", 2024-05-09T20:23:16.215 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "sysuser.client.0+test@test.test", 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "sysuser.client.0", 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "WUUFATBQMBNHMZLDLEIV", 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "VCCRmH18IqT9dUxFF8+8XxumaEs5d1AAQmOSIQTLiAE8bMSyY3dD5A==", 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:23:16.211233Z" 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:23:16.216 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "system": true, 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:23:16.217 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "rgw", 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "", 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:23:16.211191Z", 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:23:16.218 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:23:16.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.223+0000 7effaafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:23:16.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.231+0000 7f00f6f62980 20 remove_watcher() i=0 2024-05-09T20:23:16.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.231+0000 7f00f6f62980 2 removed watcher, disabling cache 2024-05-09T20:23:16.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.231+0000 7f00f6f62980 20 remove_watcher() i=1 2024-05-09T20:23:16.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.235+0000 7f00f6f62980 20 remove_watcher() i=2 2024-05-09T20:23:16.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.235+0000 7f00f6f62980 20 remove_watcher() i=3 2024-05-09T20:23:16.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.239+0000 7f00f6f62980 20 remove_watcher() i=4 2024-05-09T20:23:16.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.239+0000 7f00f6f62980 20 remove_watcher() i=5 2024-05-09T20:23:16.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.243+0000 7f00f6f62980 20 remove_watcher() i=6 2024-05-09T20:23:16.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:23:16.243+0000 7f00f6f62980 20 remove_watcher() i=7 2024-05-09T20:23:16.252 INFO:tasks.ragweed:Configuring ragweed... 2024-05-09T20:23:16.252 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:23:16.252 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/archive/ragweed.client.0.conf 2024-05-09T20:23:16.259 INFO:tasks.ragweed:Configuring boto... 2024-05-09T20:23:16.259 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:23:16.259 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/boto.cfg 2024-05-09T20:23:16.306 DEBUG:teuthology.orchestra.run.smithi063:ragweed tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/ragweed.client.0 && RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg tox --sitepackages -- -v -m 'not fails_on_rgw' 2024-05-09T20:23:16.481 INFO:teuthology.orchestra.run.smithi063.stdout:GLOB sdist-make: /home/ubuntu/cephtest/ragweed.client.0/setup.py 2024-05-09T20:23:16.665 INFO:teuthology.orchestra.run.smithi063.stdout:py create: /home/ubuntu/cephtest/ragweed.client.0/.tox/py 2024-05-09T20:23:17.236 INFO:teuthology.orchestra.run.smithi063.stdout:py installdeps: -rrequirements.txt 2024-05-09T20:23:20.671 INFO:teuthology.orchestra.run.smithi063.stdout:py inst: /home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip 2024-05-09T20:23:23.614 INFO:teuthology.orchestra.run.smithi063.stdout:py installed: appdirs==1.4.4,asyncssh==2.5.0,attrs==21.2.0,Babel==2.8.0,bcrypt==3.2.0,beautifulsoup4==4.10.0,beniget==0.4.1,blinker==1.4,boto==2.49.0,Brotli==1.0.9,cachetools==5.0.0,ceph==1.0.0,ceph-volume==1.0.0,cephfs==2.0.0,certifi==2020.6.20,chardet==4.0.0,cheroot==8.5.2+ds1,CherryPy==18.6.1,command-not-found==0.3,configobj==5.0.6,cryptography==3.4.8,cycler==0.11.0,dbus-python==1.2.18,decorator==4.4.2,distlib==0.3.4,distro==1.7.0,distro-info===1.1build1,docopt==0.6.2,execnet==1.9.0,filelock==3.6.0,fonttools==4.29.1,fs==2.4.12,gast==0.5.2,gevent==21.8.0,google-auth==1.5.1,greenlet==1.1.2,html5lib==1.1,httplib2==0.20.2,idna==3.3,importlib-metadata==4.6.4,iniconfig==1.1.1,isodate==0.6.1,jaraco.classes==3.2.1,jaraco.collections==3.4.0,jaraco.functools==3.4.0,jaraco.text==3.6.0,jeepney==0.7.1,Jinja2==3.0.3,joblib==0.17.0,keyring==23.5.0,kiwisolver==1.3.2,kubernetes==12.0.1,language-selector==0.1,launchpadlib==1.10.16,lazr.restfulclient==0.14.4,lazr.uri==1.0.6,logutils==0.3.3,lxml==4.8.0,lz4==3.1.3+dfsg,Mako==1.1.3,MarkupSafe==2.0.1,matplotlib==3.5.1,more-itertools==8.10.0,mpmath==0.0.0,munch==4.0.0,natsort==8.0.2,netifaces==0.11.0,nose==1.3.7,numpy==1.21.5,oauthlib==3.2.0,olefile==0.46,packaging==21.3,Paste==3.5.0,PasteDeploy==2.1.1,PasteScript==2.0.2,pecan==1.3.3,Pillow==9.0.1,platformdirs==2.5.1,pluggy==0.13.0,ply==3.11,portend==3.0.0,prettytable==2.5.0,psutil==5.9.0,py==1.10.0,pyasn1==0.4.8,pyasn1-modules==0.2.1,Pygments==2.11.2,PyGObject==3.42.1,pyinotify==0.9.6,PyJWT==2.3.0,pymacaroons==0.13.0,PyNaCl==1.5.0,pyOpenSSL==21.0.0,pyparsing==2.4.7,pytest==6.2.5,python-apt==2.3.0+ubuntu2.1,python-dateutil==2.8.1,python-magic==0.4.24,pythran==0.10.0,pytz==2022.1,PyYAML==5.4.1,rados==2.0.0,ragweed @ file:///home/ubuntu/cephtest/ragweed.client.0/.tox/.tmp/package/1/ragweed-0.0.1.zip#sha256=b09b96ca09808f6145f2a3df5b55da8d6b032dd204e48875e7182b078c07fee3,rbd==2.0.0,registries-conf-ctl @ git+https://github.com/sebastian-philipp/registries-conf-ctl@0e9bf2dc202aa9be9b0bf8e8265aac3f2308f9db,remoto==1.2.1,repoze.lru==0.7,requests==2.25.1,requests-oauthlib==1.3.0,rgw==2.0.0,Routes==2.5.1,rsa==4.8,s3cmd==2.2.0,scikit-learn==0.23.2,scipy==1.8.0,SecretStorage==3.3.1,simplegeneric==0.8.1,simplejson==3.17.6,six==1.16.0,soupsieve==2.3.1,ssh-import-id==5.11,sympy==1.9,Tempita==0.5.2,tempora==4.1.2,threadpoolctl==3.1.0,toml==0.10.2,ubuntu-advantage-tools==27.12,ufoLib2==0.13.1,ufw==0.36.1,unattended-upgrades==0.1,unicodedata2==14.0.0,urllib3==1.26.5,virtualenv==20.13.0+ds,wadllib==1.3.6,waitress==1.4.4,wcwidth==0.2.5,webencodings==0.5.1,WebOb==1.8.6,websocket-client==1.2.3,WebTest==2.0.35,Werkzeug==2.0.2,zc.lockfile==2.0,zipp==1.0.0,zope.event==4.4,zope.interface==5.4.0 2024-05-09T20:23:23.615 INFO:teuthology.orchestra.run.smithi063.stdout:py run-test-pre: PYTHONHASHSEED='3640841348' 2024-05-09T20:23:23.615 INFO:teuthology.orchestra.run.smithi063.stdout:py run-test: commands[0] | python -m pytest -v -m 'not fails_on_rgw' 2024-05-09T20:23:23.771 INFO:teuthology.orchestra.run.smithi063.stdout:============================= test session starts ============================== 2024-05-09T20:23:23.771 INFO:teuthology.orchestra.run.smithi063.stdout:platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /home/ubuntu/cephtest/ragweed.client.0/.tox/py/bin/python 2024-05-09T20:23:23.771 INFO:teuthology.orchestra.run.smithi063.stdout:cachedir: .tox/py/.pytest_cache 2024-05-09T20:23:23.771 INFO:teuthology.orchestra.run.smithi063.stdout:rootdir: /home/ubuntu/cephtest/ragweed.client.0, configfile: pytest.ini 2024-05-09T20:23:23.920 INFO:teuthology.orchestra.run.smithi063.stdout:collecting ... collected 9 items 2024-05-09T20:23:23.920 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:23:28.999 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_small_obj_data::test PASSED [ 11%] 2024-05-09T20:23:34.846 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_multipart_simple::test PASSED [ 22%] 2024-05-09T20:23:38.591 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_multipart_index_versioning::test PASSED [ 33%] 2024-05-09T20:23:44.336 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_multipart_defer_complete::test PASSED [ 44%] 2024-05-09T20:23:48.875 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_multipart_defer_update_complete::test PASSED [ 55%] 2024-05-09T20:24:00.470 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_obj_storage_class::test PASSED [ 66%] 2024-05-09T20:24:10.434 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart::test PASSED [ 77%] 2024-05-09T20:24:21.881 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_copy::test PASSED [ 88%] 2024-05-09T20:24:38.937 INFO:teuthology.orchestra.run.smithi063.stdout:ragweed/tests/tests.py::r_test_obj_storage_class_multipart_copy::test PASSED [100%] 2024-05-09T20:24:38.938 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:24:38.938 INFO:teuthology.orchestra.run.smithi063.stdout:=============================== warnings summary =============================== 2024-05-09T20:24:38.938 INFO:teuthology.orchestra.run.smithi063.stdout:.tox/py/lib/python3.10/site-packages/boto/plugin.py:40 2024-05-09T20:24:38.938 INFO:teuthology.orchestra.run.smithi063.stdout: /home/ubuntu/cephtest/ragweed.client.0/.tox/py/lib/python3.10/site-packages/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses 2024-05-09T20:24:38.938 INFO:teuthology.orchestra.run.smithi063.stdout: import imp 2024-05-09T20:24:38.938 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:24:38.938 INFO:teuthology.orchestra.run.smithi063.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-05-09T20:24:38.938 INFO:teuthology.orchestra.run.smithi063.stdout:=================== 9 passed, 1 warning in 75.17s (0:01:15) ==================== 2024-05-09T20:24:38.995 INFO:teuthology.orchestra.run.smithi063.stdout:___________________________________ summary ____________________________________ 2024-05-09T20:24:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: py: commands succeeded 2024-05-09T20:24:38.996 INFO:teuthology.orchestra.run.smithi063.stdout: congratulations :) 2024-05-09T20:24:39.027 INFO:tasks.ragweed:Cleaning up boto... 2024-05-09T20:24:39.027 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /home/ubuntu/cephtest/boto.cfg 2024-05-09T20:24:39.033 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2024-05-09T20:24:39.114 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:24:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:24:39.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.162+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.166+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.166+0000 7fce7ee3b980 20 realm 2024-05-09T20:24:39.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.166+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.166+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.166+0000 7fce7ee3b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:24:39.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.166+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.166+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.166+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.170+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:24:39.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.170+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.170+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:24:39.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.170+0000 7fce7ee3b980 20 searching for the correct realm 2024-05-09T20:24:39.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.174+0000 7fce7d150640 0 -- 172.21.15.63:0/2660122565 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fce70008040 msgr2=0x7fce7005bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:24:39.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.178+0000 7fce7d150640 0 -- 172.21.15.63:0/2660122565 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fcd48077490 msgr2=0x7fcd480798c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:24:39.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:24:39.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:24:39.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:24:39.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:24:39.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:24:39.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:24:39.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:24:39.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:24:39.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 zone default found 2024-05-09T20:24:39.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 4 Realm: () 2024-05-09T20:24:39.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:24:39.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:24:39.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:24:39.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 zonegroup default 2024-05-09T20:24:39.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 started sync module instance, tier type = 2024-05-09T20:24:39.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.190+0000 7fce7ee3b980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:24:39.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 20 add_watcher() i=0 2024-05-09T20:24:39.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 20 add_watcher() i=1 2024-05-09T20:24:39.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 20 add_watcher() i=2 2024-05-09T20:24:39.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 20 add_watcher() i=3 2024-05-09T20:24:39.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 20 add_watcher() i=4 2024-05-09T20:24:39.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 20 add_watcher() i=5 2024-05-09T20:24:39.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 20 add_watcher() i=6 2024-05-09T20:24:39.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 20 add_watcher() i=7 2024-05-09T20:24:39.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.210+0000 7fce7ee3b980 2 all 8 watchers are set, enabling cache 2024-05-09T20:24:39.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.214+0000 7fcd337e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:24:39.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.214+0000 7fce7ee3b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:24:39.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.214+0000 7fce7ee3b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:24:39.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.286+0000 7fcd327e4640 20 reqs_thread_entry: start 2024-05-09T20:24:39.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.286+0000 7fce7ee3b980 20 init_complete bucket index max shards: 11 2024-05-09T20:24:39.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 20 Filter name: none 2024-05-09T20:24:39.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fcd2bfff640 20 reqs_thread_entry: start 2024-05-09T20:24:39.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2024-05-09T20:24:39.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=486 2024-05-09T20:24:39.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x13 2024-05-09T20:24:39.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2024-05-09T20:24:39.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4 : miss 2024-05-09T20:24:39.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.290+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=418 2024-05-09T20:24:39.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4 info.flags=0x17 2024-05-09T20:24:39.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4 to cache LRU end 2024-05-09T20:24:39.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:39.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4 2024-05-09T20:24:39.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy 2024-05-09T20:24:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning 2024-05-09T20:24:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.294+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy 2024-05-09T20:24:39.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.298+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete 2024-05-09T20:24:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.298+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete 2024-05-09T20:24:39.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.298+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class 2024-05-09T20:24:39.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.298+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_multipart_simple 2024-05-09T20:24:39.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.298+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart 2024-05-09T20:24:39.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.302+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got tests/r_test_small_obj_data 2024-05-09T20:24:39.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.302+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x560bfb190908 s->prefetch_data=0 2024-05-09T20:24:39.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.302+0000 7fce7ee3b980 10 manifest: total_size = 7792 2024-05-09T20:24:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.302+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.5667821576538968763 2024-05-09T20:24:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.302+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x560bfb190908 s->prefetch_data=0 2024-05-09T20:24:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.306+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb190908 2024-05-09T20:24:39.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.306+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.1 2024-05-09T20:24:39.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.310+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.310+0000 7fce7ee3b980 10 manifest: total_size = 7409 2024-05-09T20:24:39.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.310+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9809690517665552624 2024-05-09T20:24:39.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.310+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.310+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11b158 2024-05-09T20:24:39.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.310+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.2 2024-05-09T20:24:39.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.314+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart_copy 2024-05-09T20:24:39.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.318+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.318+0000 7fce7ee3b980 10 manifest: total_size = 5287 2024-05-09T20:24:39.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.318+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8705752653890057167 2024-05-09T20:24:39.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.318+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.318+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11b158 2024-05-09T20:24:39.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.318+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.2 2024-05-09T20:24:39.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.322+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_index_versioning 2024-05-09T20:24:39.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.322+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.326+0000 7fce7ee3b980 10 manifest: total_size = 4978 2024-05-09T20:24:39.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.326+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13346557863626780753 2024-05-09T20:24:39.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.326+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.326+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11b158 2024-05-09T20:24:39.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.326+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.3 2024-05-09T20:24:39.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.326+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_copy 2024-05-09T20:24:39.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.330+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.330+0000 7fce7ee3b980 10 manifest: total_size = 7538 2024-05-09T20:24:39.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.330+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11735737109826908345 2024-05-09T20:24:39.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.330+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.330+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11b158 2024-05-09T20:24:39.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.330+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.4 2024-05-09T20:24:39.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.334+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_update_complete 2024-05-09T20:24:39.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.338+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.338+0000 7fce7ee3b980 10 manifest: total_size = 7630 2024-05-09T20:24:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.338+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4720825594961364050 2024-05-09T20:24:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.338+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.338+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11b158 2024-05-09T20:24:39.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.338+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.4 2024-05-09T20:24:39.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.338+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_defer_complete 2024-05-09T20:24:39.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.342+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.346+0000 7fce7ee3b980 10 manifest: total_size = 5143 2024-05-09T20:24:39.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.346+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1976734468414213072 2024-05-09T20:24:39.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.346+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.346+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11b158 2024-05-09T20:24:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.346+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.5 2024-05-09T20:24:39.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.346+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class 2024-05-09T20:24:39.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.350+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.350+0000 7fce7ee3b980 10 manifest: total_size = 5427 2024-05-09T20:24:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.350+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15070069240579865484 2024-05-09T20:24:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.350+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.350+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11b158 2024-05-09T20:24:39.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.350+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.7 2024-05-09T20:24:39.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.354+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=tests/r_test_multipart_simple 2024-05-09T20:24:39.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.358+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.358+0000 7fce7ee3b980 10 manifest: total_size = 7736 2024-05-09T20:24:39.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.358+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13556754228719087581 2024-05-09T20:24:39.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.358+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1082f0 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x560bfb11b158 s->prefetch_data=0 2024-05-09T20:24:39.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.358+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11b158 2024-05-09T20:24:39.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.358+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4.8 2024-05-09T20:24:39.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.362+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=tests/r_test_obj_storage_class_multipart 2024-05-09T20:24:39.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.366+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:39.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.366+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:39.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.366+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=tests/r_test_small_obj_data 2024-05-09T20:24:39.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.370+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:39.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.370+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:39.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.370+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:39.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.370+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:39.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.370+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:39.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.374+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2024-05-09T20:24:39.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.374+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.374+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=204 2024-05-09T20:24:39.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.374+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x11 2024-05-09T20:24:39.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.374+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2024-05-09T20:24:39.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.374+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-conf from cache 2024-05-09T20:24:39.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.374+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-conf, ofs0, ns] 2024-05-09T20:24:39.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.378+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431582 cookie 94609456629792 notifier 5615 bl.length()=162 2024-05-09T20:24:39.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.382+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4 from cache 2024-05-09T20:24:39.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.382+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-conf:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4, ofs0, ns] 2024-05-09T20:24:39.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.382+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431602 cookie 94609456637504 notifier 5615 bl.length()=219 2024-05-09T20:24:39.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.390+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-conf/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4 : miss 2024-05-09T20:24:39.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.390+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.390+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.390+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-conf/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4 info.flags=0x0 2024-05-09T20:24:39.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.390+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-conf/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4 to cache LRU end 2024-05-09T20:24:39.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.394+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11 : miss 2024-05-09T20:24:39.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.394+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.398+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=449 2024-05-09T20:24:39.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.398+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11 info.flags=0x17 2024-05-09T20:24:39.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.398+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11 to cache LRU end 2024-05-09T20:24:39.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.398+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.398+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:39.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.398+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11 2024-05-09T20:24:39.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.406+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo 2024-05-09T20:24:39.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.406+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce74008ea0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x560bfb113c88 s->prefetch_data=0 2024-05-09T20:24:39.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.406+0000 7fce7ee3b980 10 manifest: total_size = 26082 2024-05-09T20:24:39.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.406+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16571415963977778297 2024-05-09T20:24:39.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.406+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce74008ea0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x560bfb113c88 s->prefetch_data=0 2024-05-09T20:24:39.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.406+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb113c88 2024-05-09T20:24:39.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.406+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11.10 2024-05-09T20:24:39.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=3 2024-05-09T20:24:39.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=7671 2024-05-09T20:24:39.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=7671 stripe_ofs=7671 part_ofs=7671 rule->part_size=7671 2024-05-09T20:24:39.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=3 2024-05-09T20:24:39.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6299127 part_ofs=7671 rule->part_size=7671 2024-05-09T20:24:39.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=15342 stripe_ofs=15342 part_ofs=15342 rule->part_size=7673 2024-05-09T20:24:39.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=7673 rules.size()=3 2024-05-09T20:24:39.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6306798 part_ofs=15342 rule->part_size=7673 2024-05-09T20:24:39.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=23015 stripe_ofs=23015 part_ofs=23015 rule->part_size=3067 2024-05-09T20:24:39.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=3 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6314471 part_ofs=23015 rule->part_size=3067 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=26082 stripe_ofs=26082 part_ofs=26082 rule->part_size=3067 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16571415963977778297 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11__multipart_foo.2~4zRieq9ilwqK3XS8IwC6qLWIeNzb5e6.1 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 347 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11__multipart_foo.2~4zRieq9ilwqK3XS8IwC6qLWIeNzb5e6.2 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 593 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11__multipart_foo.2~4zRieq9ilwqK3XS8IwC6qLWIeNzb5e6.3 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 839 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11__multipart_foo.2~4zRieq9ilwqK3XS8IwC6qLWIeNzb5e6.4 2024-05-09T20:24:39.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1085 2024-05-09T20:24:39.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.410+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16571415963977778297 2024-05-09T20:24:39.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.414+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo 2024-05-09T20:24:39.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.414+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:39.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.414+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:39.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.418+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:39.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.418+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:39.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.418+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:39.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.418+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:39.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.418+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:39.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.422+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2024-05-09T20:24:39.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.422+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.422+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=235 2024-05-09T20:24:39.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.422+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x11 2024-05-09T20:24:39.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.422+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2024-05-09T20:24:39.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.426+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2024-05-09T20:24:39.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.426+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1, ofs0, ns] 2024-05-09T20:24:39.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.426+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431603 cookie 94609456637504 notifier 5615 bl.length()=191 2024-05-09T20:24:39.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.430+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11 from cache 2024-05-09T20:24:39.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.430+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11, ofs0, ns] 2024-05-09T20:24:39.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.434+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431567 cookie 94609456650224 notifier 5615 bl.length()=249 2024-05-09T20:24:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.438+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11 : miss 2024-05-09T20:24:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.438+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.442+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.442+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11 info.flags=0x0 2024-05-09T20:24:39.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.442+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-complete.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11 to cache LRU end 2024-05-09T20:24:39.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.446+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12 : miss 2024-05-09T20:24:39.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.446+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.446+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=456 2024-05-09T20:24:39.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.446+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12 info.flags=0x17 2024-05-09T20:24:39.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.446+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12 to cache LRU end 2024-05-09T20:24:39.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.446+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.446+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:39.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.446+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12 2024-05-09T20:24:39.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.454+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo 2024-05-09T20:24:39.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.454+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb191b80 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x560bfb121f88 s->prefetch_data=0 2024-05-09T20:24:39.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.454+0000 7fce7ee3b980 10 manifest: total_size = 18874368 2024-05-09T20:24:39.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.454+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9088638616493081612 2024-05-09T20:24:39.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.454+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb191b80 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x560bfb121f88 s->prefetch_data=0 2024-05-09T20:24:39.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.454+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb121f88 2024-05-09T20:24:39.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.454+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12.10 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=5242880 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=5242880 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=15728640 stripe_ofs=15728640 part_ofs=15728640 rule->part_size=3145728 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=3145728 rules.size()=2 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=22020096 part_ofs=15728640 rule->part_size=3145728 2024-05-09T20:24:39.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=18874368 stripe_ofs=18874368 part_ofs=18874368 rule->part_size=3145728 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9088638616493081612 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12__multipart_foo.2~Jf0RyadJWJv1w8WUBA6Yl-L1arrwOTM.1 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 346 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12__multipart_foo.2~Jf0RyadJWJv1w8WUBA6Yl-L1arrwOTM.2 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 592 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12__multipart_foo.2~Jf0RyadJWJv1w8WUBA6Yl-L1arrwOTM.3 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 838 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12__multipart_foo.2~Jf0RyadJWJv1w8WUBA6Yl-L1arrwOTM.4 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1084 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9088638616493081612 2024-05-09T20:24:39.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:39.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:39.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.462+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo 2024-05-09T20:24:39.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.466+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:39.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.466+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:39.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.466+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:39.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.466+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:39.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.466+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:39.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.470+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2024-05-09T20:24:39.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.470+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.470+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=242 2024-05-09T20:24:39.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.470+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x11 2024-05-09T20:24:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.470+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2024-05-09T20:24:39.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.474+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2024-05-09T20:24:39.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.474+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1, ofs0, ns] 2024-05-09T20:24:39.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.474+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431616 cookie 94609456642448 notifier 5615 bl.length()=198 2024-05-09T20:24:39.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.478+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12 from cache 2024-05-09T20:24:39.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.478+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12, ofs0, ns] 2024-05-09T20:24:39.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.478+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431617 cookie 94609456642448 notifier 5615 bl.length()=256 2024-05-09T20:24:39.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.486+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12 : miss 2024-05-09T20:24:39.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.486+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.486+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.486+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12 info.flags=0x0 2024-05-09T20:24:39.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.486+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-defer-update-complete.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12 to cache LRU end 2024-05-09T20:24:39.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.490+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10 : miss 2024-05-09T20:24:39.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.490+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.490+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=451 2024-05-09T20:24:39.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.490+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10 info.flags=0x17 2024-05-09T20:24:39.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.490+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10 to cache LRU end 2024-05-09T20:24:39.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.490+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.490+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:39.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.490+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10 2024-05-09T20:24:39.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.502+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo[DoqfCY4tRL24r7YUeqq7S-yWFeoS.i7] 2024-05-09T20:24:39.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.502+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10.10 2024-05-09T20:24:39.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.502+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb11c2c0 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.502+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 8mgucp5cp0jy363qi2pbho3hoalfyed2 2024-05-09T20:24:39.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.502+0000 7fce7ee3b980 20 get_obj_state_impl: setting s->olh_tag to q5djo43p88zikg00zmxv00y9cc8zxlb7 2024-05-09T20:24:39.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.502+0000 7fce7ee3b980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=q5djo43p88zikg00zmxv00y9cc8zxlb7 2024-05-09T20:24:39.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.506+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10.10 2024-05-09T20:24:39.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10.10 2024-05-09T20:24:39.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 20 olh_log_entry: epoch=3 op=2 key=foo[] 2024-05-09T20:24:39.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 20 olh_log_entry: epoch=3 op=3 key=foo[DoqfCY4tRL24r7YUeqq7S-yWFeoS.i7] 2024-05-09T20:24:39.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb11c2c0 obj=test-client.0-r-test-multipart-index-versioning.1:_:DoqfCY4tRL24r7YUeqq7S-yWFeoS.i7_foo state=0x560bfb1334e8 s->prefetch_data=0 2024-05-09T20:24:39.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 10 manifest: total_size = 1667 2024-05-09T20:24:39.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.432569573153988646 2024-05-09T20:24:39.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb11c2c0 obj=test-client.0-r-test-multipart-index-versioning.1:_:DoqfCY4tRL24r7YUeqq7S-yWFeoS.i7_foo state=0x560bfb1334e8 s->prefetch_data=0 2024-05-09T20:24:39.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb1334e8 2024-05-09T20:24:39.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.510+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10.10 2024-05-09T20:24:39.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-09T20:24:39.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-09T20:24:39.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-09T20:24:39.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-09T20:24:39.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=516 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=516 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1548 stripe_ofs=1548 part_ofs=1548 rule->part_size=119 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=119 rules.size()=2 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6293004 part_ofs=1548 rule->part_size=119 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1667 stripe_ofs=1667 part_ofs=1667 rule->part_size=119 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.432569573153988646 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10__multipart_foo.2~ZC3Rdrn4pRIE7LzOVRrqBYqxlSz8YIS.1 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 345 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10__multipart_foo.2~ZC3Rdrn4pRIE7LzOVRrqBYqxlSz8YIS.2 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 591 2024-05-09T20:24:39.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10__multipart_foo.2~ZC3Rdrn4pRIE7LzOVRrqBYqxlSz8YIS.3 2024-05-09T20:24:39.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 837 2024-05-09T20:24:39.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10__multipart_foo.2~ZC3Rdrn4pRIE7LzOVRrqBYqxlSz8YIS.4 2024-05-09T20:24:39.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1083 2024-05-09T20:24:39.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.518+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.432569573153988646 2024-05-09T20:24:39.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.522+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo[DoqfCY4tRL24r7YUeqq7S-yWFeoS.i7] 2024-05-09T20:24:39.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.522+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb11c2c0 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.526+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10.10 2024-05-09T20:24:39.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.530+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10.10 2024-05-09T20:24:39.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.530+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:39.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.530+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:39.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.534+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:39.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.534+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:39.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.534+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:39.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.534+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:39.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.534+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:39.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.534+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2024-05-09T20:24:39.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.534+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.538+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=237 2024-05-09T20:24:39.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.538+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x11 2024-05-09T20:24:39.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.538+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2024-05-09T20:24:39.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.538+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2024-05-09T20:24:39.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.538+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1, ofs0, ns] 2024-05-09T20:24:39.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.538+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431618 cookie 94609456642448 notifier 5615 bl.length()=193 2024-05-09T20:24:39.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.546+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10 from cache 2024-05-09T20:24:39.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.546+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10, ofs0, ns] 2024-05-09T20:24:39.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.546+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431583 cookie 94609456629792 notifier 5615 bl.length()=251 2024-05-09T20:24:39.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.554+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10 : miss 2024-05-09T20:24:39.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.554+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.554+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.554+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10 info.flags=0x0 2024-05-09T20:24:39.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.554+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-index-versioning.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10 to cache LRU end 2024-05-09T20:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.562+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9 : miss 2024-05-09T20:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.562+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.562+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=439 2024-05-09T20:24:39.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.562+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9 info.flags=0x17 2024-05-09T20:24:39.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.562+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9 to cache LRU end 2024-05-09T20:24:39.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.562+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.562+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:39.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.562+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9 2024-05-09T20:24:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.574+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo 2024-05-09T20:24:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.574+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb116c20 obj=test-client.0-r-test-multipart-simple.1:foo state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.574+0000 7fce7ee3b980 10 manifest: total_size = 26083 2024-05-09T20:24:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.574+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8842038008404339350 2024-05-09T20:24:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.574+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb116c20 obj=test-client.0-r-test-multipart-simple.1:foo state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.574+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11ac48 2024-05-09T20:24:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.574+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9.10 2024-05-09T20:24:39.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=4 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=7672 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=7672 stripe_ofs=7672 part_ofs=7672 rule->part_size=7673 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=7673 rules.size()=4 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6299128 part_ofs=7672 rule->part_size=7673 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=15345 stripe_ofs=15345 part_ofs=15345 rule->part_size=7671 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=4 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6306801 part_ofs=15345 rule->part_size=7671 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=23016 stripe_ofs=23016 part_ofs=23016 rule->part_size=3067 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=4 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6314472 part_ofs=23016 rule->part_size=3067 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=26083 stripe_ofs=26083 part_ofs=26083 rule->part_size=3067 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8842038008404339350 2024-05-09T20:24:39.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9__multipart_foo.2~pmYrgdQUmJNVSxioqlbeMyu3_FtSBdd.1 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 344 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9__multipart_foo.2~pmYrgdQUmJNVSxioqlbeMyu3_FtSBdd.2 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 588 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9__multipart_foo.2~pmYrgdQUmJNVSxioqlbeMyu3_FtSBdd.3 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 832 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9__multipart_foo.2~pmYrgdQUmJNVSxioqlbeMyu3_FtSBdd.4 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 1076 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.578+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.582+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8842038008404339350 2024-05-09T20:24:39.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.582+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo 2024-05-09T20:24:39.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.586+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:39.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.586+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:39.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.586+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:39.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.586+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:39.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.586+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:39.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.586+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:39.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.586+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:39.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.590+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2024-05-09T20:24:39.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.590+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.590+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=225 2024-05-09T20:24:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.590+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x11 2024-05-09T20:24:39.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.590+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2024-05-09T20:24:39.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.594+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2024-05-09T20:24:39.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.594+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-multipart-simple.1, ofs0, ns] 2024-05-09T20:24:39.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.594+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431584 cookie 94609456635376 notifier 5615 bl.length()=183 2024-05-09T20:24:39.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.602+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9 from cache 2024-05-09T20:24:39.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.602+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9, ofs0, ns] 2024-05-09T20:24:39.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.602+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431569 cookie 94609456655616 notifier 5615 bl.length()=240 2024-05-09T20:24:39.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.610+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9 : miss 2024-05-09T20:24:39.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.610+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.610+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.610+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9 info.flags=0x0 2024-05-09T20:24:39.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.610+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-multipart-simple.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9 to cache LRU end 2024-05-09T20:24:39.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.614+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15 : miss 2024-05-09T20:24:39.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.614+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.618+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=447 2024-05-09T20:24:39.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.618+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15 info.flags=0x17 2024-05-09T20:24:39.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.618+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15 to cache LRU end 2024-05-09T20:24:39.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.618+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.618+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:39.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.618+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15 2024-05-09T20:24:39.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.618+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-05-09T20:24:39.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.618+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-05-09T20:24:39.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.622+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-05-09T20:24:39.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.622+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-05-09T20:24:39.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.622+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-05-09T20:24:39.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.622+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-05-09T20:24:39.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-05-09T20:24:39.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-05-09T20:24:39.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-05-09T20:24:39.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-05-09T20:24:39.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-05-09T20:24:39.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-05-09T20:24:39.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119650 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _-nC3P-id3d4pA4w6x3ZNYqrqc3pzYfu 2024-05-09T20:24:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119650 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.626+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.0 2024-05-09T20:24:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.634+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.634+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.634+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _-nC3P-id3d4pA4w6x3ZNYqrqc3pzYfu 2024-05-09T20:24:39.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.634+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.634+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.-BJM1WK2CeZLTtbkjYXSk7T1TRlWlRp_0 2024-05-09T20:24:39.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.634+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-05-09T20:24:39.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.634+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.634+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: _-nC3P-id3d4pA4w6x3ZNYqrqc3pzYfu 2024-05-09T20:24:39.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.638+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-05-09T20:24:39.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.638+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119650 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.638+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.638+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _IMs_70PI-3ORMXRRqmA_ArTEujnJP5c 2024-05-09T20:24:39.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.638+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119650 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.638+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.638+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.2 2024-05-09T20:24:39.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.646+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.646+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.646+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _IMs_70PI-3ORMXRRqmA_ArTEujnJP5c 2024-05-09T20:24:39.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.646+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.646+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Wig8qH7dQ6JdvzDwjNl0SqCLpqWsph9_1 2024-05-09T20:24:39.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.646+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-05-09T20:24:39.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.646+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.646+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: _IMs_70PI-3ORMXRRqmA_ArTEujnJP5c 2024-05-09T20:24:39.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.650+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-05-09T20:24:39.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.650+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.650+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.650+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15408003092332821369 2024-05-09T20:24:39.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.650+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.650+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.650+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.3 2024-05-09T20:24:39.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.658+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.658+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.658+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15408003092332821369 2024-05-09T20:24:39.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.658+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.658+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_0 2024-05-09T20:24:39.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.658+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 338 2024-05-09T20:24:39.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.658+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.658+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15408003092332821369 2024-05-09T20:24:39.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.662+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-05-09T20:24:39.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.662+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce70084f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.662+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.662+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _PdWtTnQ4KJNzok_4oO9uC5w0Qr0wwKY 2024-05-09T20:24:39.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.662+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce70084f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.662+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.662+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.5 2024-05-09T20:24:39.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.670+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.670+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.670+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _PdWtTnQ4KJNzok_4oO9uC5w0Qr0wwKY 2024-05-09T20:24:39.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.670+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.670+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.OR8P3TVuvzXG1EYzlvRZDkCbXxnmiRp_0 2024-05-09T20:24:39.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.670+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-05-09T20:24:39.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.670+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.670+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _PdWtTnQ4KJNzok_4oO9uC5w0Qr0wwKY 2024-05-09T20:24:39.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.674+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.674+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-05-09T20:24:39.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.674+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.674+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _hMqa3qCqqdaHnFtNF6aVPi0Px9xwQL2 2024-05-09T20:24:39.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.674+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.674+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.674+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.5 2024-05-09T20:24:39.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.678+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.678+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.678+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _hMqa3qCqqdaHnFtNF6aVPi0Px9xwQL2 2024-05-09T20:24:39.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.678+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.678+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.IBiNlwP_mxP5o_7nKn647qQ0QewT1UJ_0 2024-05-09T20:24:39.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.678+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-05-09T20:24:39.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.678+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.678+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: _hMqa3qCqqdaHnFtNF6aVPi0Px9xwQL2 2024-05-09T20:24:39.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.682+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-05-09T20:24:39.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.682+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb120970 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.682+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.682+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9417961423076594371 2024-05-09T20:24:39.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.682+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb120970 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.682+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.682+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.5 2024-05-09T20:24:39.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9417961423076594371 2024-05-09T20:24:39.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1 2024-05-09T20:24:39.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 330 2024-05-09T20:24:39.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9417961423076594371 2024-05-09T20:24:39.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.690+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-05-09T20:24:39.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.694+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce70084f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.694+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.694+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _fZReh8AMdjyKSLvk2pi3Tel9w06FYH5 2024-05-09T20:24:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.694+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce70084f80 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.694+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.694+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.7 2024-05-09T20:24:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.702+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.702+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.702+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _fZReh8AMdjyKSLvk2pi3Tel9w06FYH5 2024-05-09T20:24:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.702+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.702+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Pwtl3naOzaDUz42V21L2lmu7uDx1UxX_1 2024-05-09T20:24:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.702+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-05-09T20:24:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.702+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.702+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: _fZReh8AMdjyKSLvk2pi3Tel9w06FYH5 2024-05-09T20:24:39.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.706+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.706+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-05-09T20:24:39.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.706+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.706+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10979833608485621736 2024-05-09T20:24:39.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.706+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.706+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.706+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.7 2024-05-09T20:24:39.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.710+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.710+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.710+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10979833608485621736 2024-05-09T20:24:39.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.710+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.710+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_0 2024-05-09T20:24:39.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.710+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 340 2024-05-09T20:24:39.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.710+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.710+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10979833608485621736 2024-05-09T20:24:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.714+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-05-09T20:24:39.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.714+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.718+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.718+0000 7fce7ee3b980 10 generate_fake_tag new tag=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1_42e85ee79d68372dee85245f7e5cb585 2024-05-09T20:24:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.718+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1_42e85ee79d68372dee85245f7e5cb585 2024-05-09T20:24:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.718+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.718+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.718+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.7 2024-05-09T20:24:39.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.722+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.722+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.722+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1_42e85ee79d68372dee85245f7e5cb585 2024-05-09T20:24:39.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.722+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.722+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1 2024-05-09T20:24:39.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.722+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 389 2024-05-09T20:24:39.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.722+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.722+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1_42e85ee79d68372dee85245f7e5cb585 2024-05-09T20:24:39.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-05-09T20:24:39.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 10 generate_fake_tag new tag=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_1_496317b00c9032648e6e6ec9882ef38a 2024-05-09T20:24:39.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_1_496317b00c9032648e6e6ec9882ef38a 2024-05-09T20:24:39.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.726+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.8 2024-05-09T20:24:39.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.734+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.734+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.734+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_1_496317b00c9032648e6e6ec9882ef38a 2024-05-09T20:24:39.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.734+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.734+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_0 2024-05-09T20:24:39.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.734+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 398 2024-05-09T20:24:39.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.734+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.734+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_1_496317b00c9032648e6e6ec9882ef38a 2024-05-09T20:24:39.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.738+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-05-09T20:24:39.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.738+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.738+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.738+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _-7foI7HZi_DqZa1tvySSpe86XRmRuGy 2024-05-09T20:24:39.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.738+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.738+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.738+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.8 2024-05-09T20:24:39.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _-7foI7HZi_DqZa1tvySSpe86XRmRuGy 2024-05-09T20:24:39.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.bxFinJddf3V7p0r-wwn2W42mScLj4jJ_0 2024-05-09T20:24:39.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-05-09T20:24:39.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: _-7foI7HZi_DqZa1tvySSpe86XRmRuGy 2024-05-09T20:24:39.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-05-09T20:24:39.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.746+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c108310 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.750+0000 7fce7ee3b980 10 manifest: total_size = 5242880 2024-05-09T20:24:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.750+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.750+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.750+0000 7fce7ee3b980 10 generate_fake_tag new tag=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_1_602aad55e8bcbd9d8ec08ab063bca4ee 2024-05-09T20:24:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.750+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_1_602aad55e8bcbd9d8ec08ab063bca4ee 2024-05-09T20:24:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.750+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c108310 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x560bfb12ad08 s->prefetch_data=0 2024-05-09T20:24:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.750+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12ad08 2024-05-09T20:24:39.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.750+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15.9 2024-05-09T20:24:39.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.754+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.754+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.754+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_1_602aad55e8bcbd9d8ec08ab063bca4ee 2024-05-09T20:24:39.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.754+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.754+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_0 2024-05-09T20:24:39.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.754+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 396 2024-05-09T20:24:39.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.754+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.754+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_1_602aad55e8bcbd9d8ec08ab063bca4ee 2024-05-09T20:24:39.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.758+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-05-09T20:24:39.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.758+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:39.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.758+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:39.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.762+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:39.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.762+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:39.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.762+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:39.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.762+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:39.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.762+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:39.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.766+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2024-05-09T20:24:39.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.766+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.766+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=233 2024-05-09T20:24:39.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.766+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x11 2024-05-09T20:24:39.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.766+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2024-05-09T20:24:39.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.770+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2024-05-09T20:24:39.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.770+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1, ofs0, ns] 2024-05-09T20:24:39.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.770+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431619 cookie 94609456642448 notifier 5615 bl.length()=189 2024-05-09T20:24:39.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.778+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15 from cache 2024-05-09T20:24:39.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.778+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15, ofs0, ns] 2024-05-09T20:24:39.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.778+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431585 cookie 94609456629792 notifier 5615 bl.length()=247 2024-05-09T20:24:39.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.786+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15 : miss 2024-05-09T20:24:39.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.786+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.786+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.786+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15 info.flags=0x0 2024-05-09T20:24:39.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.786+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-copy.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15 to cache LRU end 2024-05-09T20:24:39.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.790+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16 : miss 2024-05-09T20:24:39.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.790+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.794+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=457 2024-05-09T20:24:39.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.794+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16 info.flags=0x17 2024-05-09T20:24:39.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.794+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16 to cache LRU end 2024-05-09T20:24:39.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.794+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.794+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:39.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.794+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16 2024-05-09T20:24:39.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.794+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM 2024-05-09T20:24:39.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.798+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD 2024-05-09T20:24:39.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.798+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-05-09T20:24:39.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.798+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM 2024-05-09T20:24:39.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.798+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN 2024-05-09T20:24:39.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.798+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-05-09T20:24:39.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN-STANDARD 2024-05-09T20:24:39.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-05-09T20:24:39.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD-STANDARD 2024-05-09T20:24:39.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM 2024-05-09T20:24:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD-FROZEN 2024-05-09T20:24:39.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN-FROZEN 2024-05-09T20:24:39.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x560bfb1178a8 s->prefetch_data=0 2024-05-09T20:24:39.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 10 manifest: total_size = 1087 2024-05-09T20:24:39.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _KiAzbfH57ywPnGbhYvUHWlrJAn5nC57 2024-05-09T20:24:39.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x560bfb1178a8 s->prefetch_data=0 2024-05-09T20:24:39.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb1178a8 2024-05-09T20:24:39.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.802+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.0 2024-05-09T20:24:39.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.810+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.810+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.810+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _KiAzbfH57ywPnGbhYvUHWlrJAn5nC57 2024-05-09T20:24:39.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.810+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.810+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.axjOseD9_5CgGQN_u-Nm0ZcacfKWtee_0 2024-05-09T20:24:39.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.810+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-05-09T20:24:39.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.810+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.810+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: _KiAzbfH57ywPnGbhYvUHWlrJAn5nC57 2024-05-09T20:24:39.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.814+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea1d0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x560bfb117888 s->prefetch_data=0 2024-05-09T20:24:39.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.814+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-LUKEWARM 2024-05-09T20:24:39.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.814+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.814+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _X7Kom-jvycD1c3G5Bcw7md2QMH0ANoj 2024-05-09T20:24:39.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.814+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea1d0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x560bfb117888 s->prefetch_data=0 2024-05-09T20:24:39.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.814+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb117888 2024-05-09T20:24:39.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.814+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.2 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _X7Kom-jvycD1c3G5Bcw7md2QMH0ANoj 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.lglYukseiKAFeOr7UdB7nDb6A3SM6bj_1 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.lglYukseiKAFeOr7UdB7nDb6A3SM6bj_2 2024-05-09T20:24:39.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-05-09T20:24:39.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.822+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _X7Kom-jvycD1c3G5Bcw7md2QMH0ANoj 2024-05-09T20:24:39.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.826+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb121440 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x560bfb117888 s->prefetch_data=0 2024-05-09T20:24:39.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.826+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-STANDARD 2024-05-09T20:24:39.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.826+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.826+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11107501496804072710 2024-05-09T20:24:39.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.826+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb121440 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x560bfb117888 s->prefetch_data=0 2024-05-09T20:24:39.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.826+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb117888 2024-05-09T20:24:39.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.826+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.3 2024-05-09T20:24:39.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11107501496804072710 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.1 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 368 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2 2024-05-09T20:24:39.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 635 2024-05-09T20:24:39.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.3 2024-05-09T20:24:39.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 902 2024-05-09T20:24:39.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11107501496804072710 2024-05-09T20:24:39.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-05-09T20:24:39.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.834+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x560bfb125868 s->prefetch_data=0 2024-05-09T20:24:39.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.838+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.838+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _boKDksGCANDFfLfNxv2ljhfOKWKmJjx 2024-05-09T20:24:39.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.838+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x560bfb125868 s->prefetch_data=0 2024-05-09T20:24:39.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.838+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb125868 2024-05-09T20:24:39.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.838+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.5 2024-05-09T20:24:39.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _boKDksGCANDFfLfNxv2ljhfOKWKmJjx 2024-05-09T20:24:39.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.tUqJDp0FiphJZ62UGdNG8m_37Jgmi2R_0 2024-05-09T20:24:39.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 310 2024-05-09T20:24:39.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.tUqJDp0FiphJZ62UGdNG8m_37Jgmi2R_1 2024-05-09T20:24:39.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 549 2024-05-09T20:24:39.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.842+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: _boKDksGCANDFfLfNxv2ljhfOKWKmJjx 2024-05-09T20:24:39.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.846+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-FROZEN-LUKEWARM 2024-05-09T20:24:39.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.846+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x560bfb125868 s->prefetch_data=0 2024-05-09T20:24:39.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.846+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.846+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _SllOnU4zdLyyMQJMPprNV36hT0qkTxE 2024-05-09T20:24:39.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.846+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x560bfb125868 s->prefetch_data=0 2024-05-09T20:24:39.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.846+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb125868 2024-05-09T20:24:39.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.846+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.5 2024-05-09T20:24:39.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _SllOnU4zdLyyMQJMPprNV36hT0qkTxE 2024-05-09T20:24:39.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.NepXrdEsrS37EwU3YFmQ_g2KcSKdpH__0 2024-05-09T20:24:39.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-05-09T20:24:39.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.NepXrdEsrS37EwU3YFmQ_g2KcSKdpH__1 2024-05-09T20:24:39.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 545 2024-05-09T20:24:39.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _SllOnU4zdLyyMQJMPprNV36hT0qkTxE 2024-05-09T20:24:39.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-FROZEN 2024-05-09T20:24:39.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.854+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.858+0000 7fce7ee3b980 10 manifest: total_size = 1087 2024-05-09T20:24:39.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.858+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15786862865251314352 2024-05-09T20:24:39.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.858+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.858+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11ac48 2024-05-09T20:24:39.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.858+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.5 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=55 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=55 rules.size()=2 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=55 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=1087 rule->part_size=55 2024-05-09T20:24:39.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15786862865251314352 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.1 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 629 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.3 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 893 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.862+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15786862865251314352 2024-05-09T20:24:39.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.866+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-05-09T20:24:39.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.866+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119ec0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.866+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.866+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _Ad7Xqyy-25c_rN7GA5OU8Jlo9gioJUK 2024-05-09T20:24:39.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.866+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119ec0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.866+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11ac48 2024-05-09T20:24:39.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.866+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.7 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _Ad7Xqyy-25c_rN7GA5OU8Jlo9gioJUK 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.-kcz68wsn5PSN-GbyWRTmupWoY-gbig_1 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 301 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.-kcz68wsn5PSN-GbyWRTmupWoY-gbig_2 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 531 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.874+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _Ad7Xqyy-25c_rN7GA5OU8Jlo9gioJUK 2024-05-09T20:24:39.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.878+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.878+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN-STANDARD 2024-05-09T20:24:39.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.878+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.878+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.70945469219796654 2024-05-09T20:24:39.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.878+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.878+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11ac48 2024-05-09T20:24:39.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.878+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.7 2024-05-09T20:24:39.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.70945469219796654 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.1 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 371 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2 2024-05-09T20:24:39.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 644 2024-05-09T20:24:39.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.3 2024-05-09T20:24:39.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 917 2024-05-09T20:24:39.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.70945469219796654 2024-05-09T20:24:39.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb11d7a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.886+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 10 manifest: total_size = 1087 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 10 generate_fake_tag new tag=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2_24e4605941622e1c8ec9e4771ba59838 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2_24e4605941622e1c8ec9e4771ba59838 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb11d7a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11ac48 2024-05-09T20:24:39.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.890+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.7 2024-05-09T20:24:39.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=516 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=516 stripe_ofs=516 part_ofs=516 rule->part_size=516 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=516 rules.size()=2 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291972 part_ofs=516 rule->part_size=516 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1032 stripe_ofs=1032 part_ofs=1032 rule->part_size=55 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=55 rules.size()=2 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6292488 part_ofs=1032 rule->part_size=55 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=1087 rule->part_size=55 2024-05-09T20:24:39.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2_24e4605941622e1c8ec9e4771ba59838 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.1 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 440 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 704 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.3 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 968 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.894+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2_24e4605941622e1c8ec9e4771ba59838 2024-05-09T20:24:39.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-STANDARD 2024-05-09T20:24:39.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 10 generate_fake_tag new tag=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2_8494cc01ca0c51c8ecc9c910e4f7d676 2024-05-09T20:24:39.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2_8494cc01ca0c51c8ecc9c910e4f7d676 2024-05-09T20:24:39.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce7005c470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11ac48 2024-05-09T20:24:39.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.898+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.8 2024-05-09T20:24:39.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-09T20:24:39.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2_8494cc01ca0c51c8ecc9c910e4f7d676 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.1 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 449 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 722 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.3 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 995 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2_8494cc01ca0c51c8ecc9c910e4f7d676 2024-05-09T20:24:39.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.906+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-LUKEWARM-LUKEWARM 2024-05-09T20:24:39.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.910+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb116c70 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.910+0000 7fce7ee3b980 10 manifest: total_size = 1087 2024-05-09T20:24:39.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.910+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to _fEyVgs89_z0CBqYk48IpiWhPAvqvtQY 2024-05-09T20:24:39.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.910+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb116c70 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.910+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11ac48 2024-05-09T20:24:39.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.910+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.8 2024-05-09T20:24:39.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.914+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:39.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.914+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=1087 stripe_ofs=1087 part_ofs=0 rule->part_size=0 2024-05-09T20:24:39.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.914+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: _fEyVgs89_z0CBqYk48IpiWhPAvqvtQY 2024-05-09T20:24:39.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.914+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.914+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.nxA66Srw8M-PCSXd8BURxC_LpT7MZKM_0 2024-05-09T20:24:39.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.914+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 308 2024-05-09T20:24:39.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.914+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.914+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: _fEyVgs89_z0CBqYk48IpiWhPAvqvtQY 2024-05-09T20:24:39.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD-FROZEN 2024-05-09T20:24:39.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 10 generate_fake_tag new tag=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2_6c4622c91d7effb4b3496e9f7f872839 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2_6c4622c91d7effb4b3496e9f7f872839 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c1212a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x560bfb11ac48 s->prefetch_data=0 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb11ac48 2024-05-09T20:24:39.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.918+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16.9 2024-05-09T20:24:39.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2_6c4622c91d7effb4b3496e9f7f872839 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.1 2024-05-09T20:24:39.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 441 2024-05-09T20:24:39.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2 2024-05-09T20:24:39.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 708 2024-05-09T20:24:39.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.3 2024-05-09T20:24:39.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 975 2024-05-09T20:24:39.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2_6c4622c91d7effb4b3496e9f7f872839 2024-05-09T20:24:39.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.926+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-FROZEN-FROZEN 2024-05-09T20:24:39.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.930+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:39.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.930+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:39.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.930+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:39.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.930+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:39.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.930+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:39.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.930+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:39.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.930+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:39.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.934+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2024-05-09T20:24:39.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.934+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.938+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=243 2024-05-09T20:24:39.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.938+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x11 2024-05-09T20:24:39.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.938+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2024-05-09T20:24:39.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.942+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2024-05-09T20:24:39.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.942+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1, ofs0, ns] 2024-05-09T20:24:39.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.942+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431570 cookie 94609456639632 notifier 5615 bl.length()=199 2024-05-09T20:24:39.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.946+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16 from cache 2024-05-09T20:24:39.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.946+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16, ofs0, ns] 2024-05-09T20:24:39.942 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.950+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431571 cookie 94609456639632 notifier 5615 bl.length()=257 2024-05-09T20:24:39.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.958+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16 : miss 2024-05-09T20:24:39.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.958+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.958+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:39.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.958+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16 info.flags=0x0 2024-05-09T20:24:39.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.958+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart-copy.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16 to cache LRU end 2024-05-09T20:24:39.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.962+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14 : miss 2024-05-09T20:24:39.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.962+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:39.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.966+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=452 2024-05-09T20:24:39.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.966+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14 info.flags=0x17 2024-05-09T20:24:39.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.966+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14 to cache LRU end 2024-05-09T20:24:39.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.966+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:39.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.966+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:39.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.966+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14 2024-05-09T20:24:39.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.966+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-05-09T20:24:39.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.970+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-05-09T20:24:39.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.970+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-05-09T20:24:39.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.974+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb132be0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x560bfaf89da8 s->prefetch_data=0 2024-05-09T20:24:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.974+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.974+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.60331768207756217 2024-05-09T20:24:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.974+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb132be0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x560bfaf89da8 s->prefetch_data=0 2024-05-09T20:24:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.974+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfaf89da8 2024-05-09T20:24:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.974+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14.3 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.60331768207756217 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-FROZEN.2~x0I2e2ukALgDWftEc_XFCDuQBc_Wl3R.1 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 365 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-FROZEN.2~x0I2e2ukALgDWftEc_XFCDuQBc_Wl3R.2 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 632 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-FROZEN.2~x0I2e2ukALgDWftEc_XFCDuQBc_Wl3R.3 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 899 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.977 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.982+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.60331768207756217 2024-05-09T20:24:39.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.986+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-05-09T20:24:39.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.986+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb12e7c0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x560bfaf89da8 s->prefetch_data=0 2024-05-09T20:24:39.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.986+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.986+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.2657146615741495147 2024-05-09T20:24:39.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.986+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb12e7c0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x560bfaf89da8 s->prefetch_data=0 2024-05-09T20:24:39.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.986+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfaf89da8 2024-05-09T20:24:39.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.986+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14.5 2024-05-09T20:24:39.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.2657146615741495147 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-STANDARD.2~iRwd8HoES5HWxG0iadvGkVD-XMW16Wg.1 2024-05-09T20:24:39.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 364 2024-05-09T20:24:39.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-STANDARD.2~iRwd8HoES5HWxG0iadvGkVD-XMW16Wg.2 2024-05-09T20:24:39.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 628 2024-05-09T20:24:39.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-STANDARD.2~iRwd8HoES5HWxG0iadvGkVD-XMW16Wg.3 2024-05-09T20:24:39.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 892 2024-05-09T20:24:39.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.994+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.2657146615741495147 2024-05-09T20:24:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.998+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-05-09T20:24:39.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.998+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb120900 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x560bfaf89da8 s->prefetch_data=0 2024-05-09T20:24:39.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.998+0000 7fce7ee3b980 10 manifest: total_size = 11534336 2024-05-09T20:24:39.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.998+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17973264292651109368 2024-05-09T20:24:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.998+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb120900 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x560bfaf89da8 s->prefetch_data=0 2024-05-09T20:24:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.998+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfaf89da8 2024-05-09T20:24:39.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:39.998+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14.7 2024-05-09T20:24:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=6291456 part_ofs=0 rule->part_size=5242880 2024-05-09T20:24:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=11534336 part_ofs=5242880 rule->part_size=5242880 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): stripe_ofs=16777216 part_ofs=10485760 rule->part_size=1048576 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17973264292651109368 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-LUKEWARM.2~JEfPmSaHqZPRt0JGI2OsSBRqZsFhW-f.1 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 374 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-LUKEWARM.2~JEfPmSaHqZPRt0JGI2OsSBRqZsFhW-f.2 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 647 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-LUKEWARM.2~JEfPmSaHqZPRt0JGI2OsSBRqZsFhW-f.3 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 920 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:39.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17973264292651109368 2024-05-09T20:24:40.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:40.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:40.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.006+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-05-09T20:24:40.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.010+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:40.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.010+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:40.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.010+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:40.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.010+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:40.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.010+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:40.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.014+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2024-05-09T20:24:40.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.014+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.014+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=238 2024-05-09T20:24:40.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.014+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x11 2024-05-09T20:24:40.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.014+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2024-05-09T20:24:40.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.018+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2024-05-09T20:24:40.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.018+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1, ofs0, ns] 2024-05-09T20:24:40.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.018+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431568 cookie 94609456650224 notifier 5615 bl.length()=194 2024-05-09T20:24:40.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.026+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14 from cache 2024-05-09T20:24:40.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.026+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14, ofs0, ns] 2024-05-09T20:24:40.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.026+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431572 cookie 94609456639632 notifier 5615 bl.length()=252 2024-05-09T20:24:40.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.034+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14 : miss 2024-05-09T20:24:40.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.034+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.034+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.034+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14 info.flags=0x0 2024-05-09T20:24:40.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.034+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class-multipart.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14 to cache LRU end 2024-05-09T20:24:40.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.042+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13 : miss 2024-05-09T20:24:40.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.042+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.042+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=442 2024-05-09T20:24:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.042+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13 info.flags=0x17 2024-05-09T20:24:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.042+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13 to cache LRU end 2024-05-09T20:24:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.042+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.042+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:40.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.042+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13 2024-05-09T20:24:40.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.046+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-FROZEN 2024-05-09T20:24:40.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.046+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-STANDARD 2024-05-09T20:24:40.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.050+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got foo-LUKEWARM 2024-05-09T20:24:40.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.050+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb120900 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x560bfaf8afc8 s->prefetch_data=0 2024-05-09T20:24:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.054+0000 7fce7ee3b980 10 manifest: total_size = 10485760 2024-05-09T20:24:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.054+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8681860532371221348 2024-05-09T20:24:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.054+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb120900 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x560bfaf8afc8 s->prefetch_data=0 2024-05-09T20:24:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.054+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfaf8afc8 2024-05-09T20:24:40.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.054+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13.3 2024-05-09T20:24:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:40.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-05-09T20:24:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-05-09T20:24:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8681860532371221348 2024-05-09T20:24:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13__shadow_.6KpARd4yTGR18HEhKqAoThCxMgGj6J9_0 2024-05-09T20:24:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 337 2024-05-09T20:24:40.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13__shadow_.6KpARd4yTGR18HEhKqAoThCxMgGj6J9_1 2024-05-09T20:24:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 574 2024-05-09T20:24:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:40.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.058+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8681860532371221348 2024-05-09T20:24:40.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.062+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-FROZEN 2024-05-09T20:24:40.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.062+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce74055040 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x560bfaf8afc8 s->prefetch_data=0 2024-05-09T20:24:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.066+0000 7fce7ee3b980 10 manifest: total_size = 649292 2024-05-09T20:24:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.066+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11706523976110306596 2024-05-09T20:24:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.066+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce74055040 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x560bfaf8afc8 s->prefetch_data=0 2024-05-09T20:24:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.066+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfaf8afc8 2024-05-09T20:24:40.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.066+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13.5 2024-05-09T20:24:40.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.070+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c0f1ea0 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.074+0000 7fce7ee3b980 10 manifest: total_size = 10485760 2024-05-09T20:24:40.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.074+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13676252935118820155 2024-05-09T20:24:40.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.074+0000 7fce7ee3b980 20 get_obj_state: octx=0x7fce6c0f1ea0 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.074+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12dbf8 2024-05-09T20:24:40.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.074+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13.7 2024-05-09T20:24:40.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.074+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=foo-STANDARD 2024-05-09T20:24:40.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:40.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=6291456 stripe_ofs=6291456 part_ofs=0 rule->part_size=0 2024-05-09T20:24:40.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2024-05-09T20:24:40.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2024-05-09T20:24:40.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13676252935118820155 2024-05-09T20:24:40.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - rgw_max_chunk_size is: 4194304 2024-05-09T20:24:40.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13__shadow_.XUSNQwpZdMvb2Bb90ia6m6miTR_QQ2t_0 2024-05-09T20:24:40.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 340 2024-05-09T20:24:40.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - adding obj with name: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13__shadow_.XUSNQwpZdMvb2Bb90ia6m6miTR_QQ2t_1 2024-05-09T20:24:40.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - total_encoded_size is: 579 2024-05-09T20:24:40.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_split_chain - sending leftover objects 2024-05-09T20:24:40.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.078+0000 7fce7ee3b980 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13676252935118820155 2024-05-09T20:24:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.082+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=foo-LUKEWARM 2024-05-09T20:24:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.082+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:40.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.082+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.086+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.086+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.086+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.086+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:40.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.086+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.090+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2024-05-09T20:24:40.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.090+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.094+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=228 2024-05-09T20:24:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.094+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x11 2024-05-09T20:24:40.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.094+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2024-05-09T20:24:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.094+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2024-05-09T20:24:40.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.094+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1, ofs0, ns] 2024-05-09T20:24:40.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.094+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431620 cookie 94609456642448 notifier 5615 bl.length()=184 2024-05-09T20:24:40.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.102+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13 from cache 2024-05-09T20:24:40.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.102+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13, ofs0, ns] 2024-05-09T20:24:40.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.102+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431621 cookie 94609456642448 notifier 5615 bl.length()=242 2024-05-09T20:24:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.114+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13 : miss 2024-05-09T20:24:40.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.114+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.114+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.114+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13 info.flags=0x0 2024-05-09T20:24:40.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.114+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-obj-storage-class.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13 to cache LRU end 2024-05-09T20:24:40.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.122+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6 : miss 2024-05-09T20:24:40.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.122+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.122+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=437 2024-05-09T20:24:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.122+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6 info.flags=0x17 2024-05-09T20:24:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.122+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6 to cache LRU end 2024-05-09T20:24:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.122+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.122+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:40.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.122+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6 2024-05-09T20:24:40.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.126+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got ___ 2024-05-09T20:24:40.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.126+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got __ _ 2024-05-09T20:24:40.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.130+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got __ 2024-05-09T20:24:40.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.130+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got obj 2024-05-09T20:24:40.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.134+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea450 obj=test-client.0-r-test-small-obj-data.1:___ state=0x560bfb1174c8 s->prefetch_data=0 2024-05-09T20:24:40.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.134+0000 7fce7ee3b980 10 manifest: total_size = 0 2024-05-09T20:24:40.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.134+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9266524097792687015 2024-05-09T20:24:40.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.134+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea450 obj=test-client.0-r-test-small-obj-data.1:___ state=0x560bfb1174c8 s->prefetch_data=0 2024-05-09T20:24:40.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.134+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb1174c8 2024-05-09T20:24:40.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.134+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6.2 2024-05-09T20:24:40.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.142+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea450 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x560bfb1174c8 s->prefetch_data=0 2024-05-09T20:24:40.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.142+0000 7fce7ee3b980 10 manifest: total_size = 0 2024-05-09T20:24:40.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.142+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.5888876148454892796 2024-05-09T20:24:40.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.142+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea450 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x560bfb1174c8 s->prefetch_data=0 2024-05-09T20:24:40.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.142+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb1174c8 2024-05-09T20:24:40.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.142+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6.3 2024-05-09T20:24:40.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.146+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=___ 2024-05-09T20:24:40.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.150+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea450 obj=test-client.0-r-test-small-obj-data.1:__ state=0x560bfb1174c8 s->prefetch_data=0 2024-05-09T20:24:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.150+0000 7fce7ee3b980 10 manifest: total_size = 0 2024-05-09T20:24:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.150+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14768777072358465043 2024-05-09T20:24:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.150+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea450 obj=test-client.0-r-test-small-obj-data.1:__ state=0x560bfb1174c8 s->prefetch_data=0 2024-05-09T20:24:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.150+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb1174c8 2024-05-09T20:24:40.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.150+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6.6 2024-05-09T20:24:40.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.154+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=__ _ 2024-05-09T20:24:40.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.158+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea450 obj=test-client.0-r-test-small-obj-data.1:obj state=0x560bfb1174c8 s->prefetch_data=0 2024-05-09T20:24:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.158+0000 7fce7ee3b980 10 manifest: total_size = 0 2024-05-09T20:24:40.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.158+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1917559096987957784 2024-05-09T20:24:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.158+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0ea450 obj=test-client.0-r-test-small-obj-data.1:obj state=0x560bfb1174c8 s->prefetch_data=0 2024-05-09T20:24:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.158+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb1174c8 2024-05-09T20:24:40.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.158+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6.8 2024-05-09T20:24:40.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.162+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=__ 2024-05-09T20:24:40.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.166+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:40.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.166+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:40.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.170+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=obj 2024-05-09T20:24:40.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.174+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:40.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.174+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:40.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.174+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:40.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.174+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:40.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.174+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:40.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.178+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2024-05-09T20:24:40.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.178+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.178+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=223 2024-05-09T20:24:40.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.178+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x11 2024-05-09T20:24:40.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.178+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2024-05-09T20:24:40.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.182+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2024-05-09T20:24:40.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.182+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.1, ofs0, ns] 2024-05-09T20:24:40.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.182+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431604 cookie 94609456637504 notifier 5615 bl.length()=181 2024-05-09T20:24:40.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.190+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6 from cache 2024-05-09T20:24:40.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.190+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6, ofs0, ns] 2024-05-09T20:24:40.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.190+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431586 cookie 94609456635376 notifier 5615 bl.length()=238 2024-05-09T20:24:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.206+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6 : miss 2024-05-09T20:24:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.206+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.206+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.206+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6 info.flags=0x0 2024-05-09T20:24:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.206+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.1/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6 to cache LRU end 2024-05-09T20:24:40.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.214+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7 : miss 2024-05-09T20:24:40.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.214+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.214+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=437 2024-05-09T20:24:40.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.214+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7 info.flags=0x17 2024-05-09T20:24:40.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.214+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7 to cache LRU end 2024-05-09T20:24:40.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.214+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:40.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.214+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:40.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.214+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7 2024-05-09T20:24:40.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.218+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got ___ 2024-05-09T20:24:40.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.218+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got __ _ 2024-05-09T20:24:40.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.222+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got __ 2024-05-09T20:24:40.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.226+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got obj 2024-05-09T20:24:40.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.226+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119ec0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x560bfb13a138 s->prefetch_data=0 2024-05-09T20:24:40.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.230+0000 7fce7ee3b980 10 manifest: total_size = 527 2024-05-09T20:24:40.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.230+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.688511657888837408 2024-05-09T20:24:40.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.230+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119ec0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x560bfb13a138 s->prefetch_data=0 2024-05-09T20:24:40.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.230+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb13a138 2024-05-09T20:24:40.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.230+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7.2 2024-05-09T20:24:40.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.234+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119470 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.238+0000 7fce7ee3b980 10 manifest: total_size = 38 2024-05-09T20:24:40.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.238+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15249897782436946575 2024-05-09T20:24:40.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.238+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb119470 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.238+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12dbf8 2024-05-09T20:24:40.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.238+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7.3 2024-05-09T20:24:40.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.238+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=___ 2024-05-09T20:24:40.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.242+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb122c80 obj=test-client.0-r-test-small-obj-data.2:__ state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.246+0000 7fce7ee3b980 10 manifest: total_size = 2078 2024-05-09T20:24:40.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.246+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.513497552200130012 2024-05-09T20:24:40.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.246+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb122c80 obj=test-client.0-r-test-small-obj-data.2:__ state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.246+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12dbf8 2024-05-09T20:24:40.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.246+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7.6 2024-05-09T20:24:40.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.246+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=__ _ 2024-05-09T20:24:40.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.254+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb122c80 obj=test-client.0-r-test-small-obj-data.2:obj state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.254+0000 7fce7ee3b980 10 manifest: total_size = 524288 2024-05-09T20:24:40.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.254+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6979262137605876050 2024-05-09T20:24:40.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.254+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb122c80 obj=test-client.0-r-test-small-obj-data.2:obj state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.254+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12dbf8 2024-05-09T20:24:40.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.254+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7.8 2024-05-09T20:24:40.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.254+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=__ 2024-05-09T20:24:40.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.258+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:40.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.258+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:40.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.262+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=obj 2024-05-09T20:24:40.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.266+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:40.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.266+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:40.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.266+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:40.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.266+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:40.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.266+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:40.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.270+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2024-05-09T20:24:40.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.270+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.270+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=223 2024-05-09T20:24:40.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.270+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x11 2024-05-09T20:24:40.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.270+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2024-05-09T20:24:40.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.274+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2024-05-09T20:24:40.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.274+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.2, ofs0, ns] 2024-05-09T20:24:40.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.274+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431605 cookie 94609456637504 notifier 5615 bl.length()=181 2024-05-09T20:24:40.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.282+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7 from cache 2024-05-09T20:24:40.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.282+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7, ofs0, ns] 2024-05-09T20:24:40.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.282+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431587 cookie 94609456635376 notifier 5615 bl.length()=238 2024-05-09T20:24:40.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.290+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7 : miss 2024-05-09T20:24:40.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.290+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.294+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.294+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7 info.flags=0x0 2024-05-09T20:24:40.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.294+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.2/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7 to cache LRU end 2024-05-09T20:24:40.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.298+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8 : miss 2024-05-09T20:24:40.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.298+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.298+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=437 2024-05-09T20:24:40.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.298+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8 info.flags=0x17 2024-05-09T20:24:40.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.298+0000 7fce7ee3b980 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8 to cache LRU end 2024-05-09T20:24:40.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.298+0000 7fce7ee3b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:40.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.298+0000 7fce7ee3b980 10 updating xattr: name=user.rgw.acl bl.length()=209 2024-05-09T20:24:40.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.298+0000 7fce7ee3b980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8 2024-05-09T20:24:40.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.302+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got ___ 2024-05-09T20:24:40.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.302+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got __ _ 2024-05-09T20:24:40.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.306+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got __ 2024-05-09T20:24:40.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.306+0000 7fce7ee3b980 10 cls_bucket_list_unordered: got obj 2024-05-09T20:24:40.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.310+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0e94c0 obj=test-client.0-r-test-small-obj-data.3:___ state=0x560bfaf8ac48 s->prefetch_data=0 2024-05-09T20:24:40.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.310+0000 7fce7ee3b980 10 manifest: total_size = 49187 2024-05-09T20:24:40.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.310+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17095183090979824149 2024-05-09T20:24:40.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.310+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0e94c0 obj=test-client.0-r-test-small-obj-data.3:___ state=0x560bfaf8ac48 s->prefetch_data=0 2024-05-09T20:24:40.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.310+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfaf8ac48 2024-05-09T20:24:40.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.310+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8.2 2024-05-09T20:24:40.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.318+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0e94c0 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x560bfaf8ac48 s->prefetch_data=0 2024-05-09T20:24:40.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.318+0000 7fce7ee3b980 10 manifest: total_size = 55 2024-05-09T20:24:40.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.318+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.237374621869548067 2024-05-09T20:24:40.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.318+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0e94c0 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x560bfaf8ac48 s->prefetch_data=0 2024-05-09T20:24:40.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.318+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfaf8ac48 2024-05-09T20:24:40.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.318+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8.3 2024-05-09T20:24:40.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.318+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=___ 2024-05-09T20:24:40.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.322+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0e94c0 obj=test-client.0-r-test-small-obj-data.3:__ state=0x560bfb129628 s->prefetch_data=0 2024-05-09T20:24:40.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.326+0000 7fce7ee3b980 10 manifest: total_size = 49187 2024-05-09T20:24:40.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.326+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17938529235184360821 2024-05-09T20:24:40.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.326+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0e94c0 obj=test-client.0-r-test-small-obj-data.3:__ state=0x560bfb129628 s->prefetch_data=0 2024-05-09T20:24:40.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.326+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb129628 2024-05-09T20:24:40.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.326+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8.6 2024-05-09T20:24:40.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.326+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=__ _ 2024-05-09T20:24:40.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.330+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0e94c0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.334+0000 7fce7ee3b980 10 manifest: total_size = 55 2024-05-09T20:24:40.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.334+0000 7fce7ee3b980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8883336963283582072 2024-05-09T20:24:40.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.334+0000 7fce7ee3b980 20 get_obj_state: octx=0x560bfb0e94c0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x560bfb12dbf8 s->prefetch_data=0 2024-05-09T20:24:40.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.334+0000 7fce7ee3b980 20 prepare_atomic_modification: state is not atomic. state=0x560bfb12dbf8 2024-05-09T20:24:40.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.334+0000 7fce7ee3b980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8.8 2024-05-09T20:24:40.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.334+0000 7fcd537fe640 20 handle_completion(): completion ok for obj=__ 2024-05-09T20:24:40.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.338+0000 7fce7ee3b980 20 list_objects_ordered: starting attempt 1 2024-05-09T20:24:40.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.338+0000 7fce7ee3b980 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-09T20:24:40.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.342+0000 7fcd53fff640 20 handle_completion(): completion ok for obj=obj 2024-05-09T20:24:40.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.342+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-09T20:24:40.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.342+0000 7fce7ee3b980 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-09T20:24:40.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.342+0000 7fce7ee3b980 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-09T20:24:40.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.342+0000 7fce7ee3b980 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-09T20:24:40.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.342+0000 7fce7ee3b980 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8])"; uploads.size()=0; is_truncated=0 2024-05-09T20:24:40.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.346+0000 7fce7ee3b980 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2024-05-09T20:24:40.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.346+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.346+0000 7fce7ee3b980 20 rados_obj.operate() r=0 bl.length=223 2024-05-09T20:24:40.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.346+0000 7fce7ee3b980 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x11 2024-05-09T20:24:40.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.346+0000 7fce7ee3b980 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2024-05-09T20:24:40.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.350+0000 7fce7ee3b980 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2024-05-09T20:24:40.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.350+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:test-client.0-r-test-small-obj-data.3, ofs0, ns] 2024-05-09T20:24:40.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.350+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431606 cookie 94609456637504 notifier 5615 bl.length()=181 2024-05-09T20:24:40.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.358+0000 7fce7ee3b980 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8 from cache 2024-05-09T20:24:40.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.358+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8, ofs0, ns] 2024-05-09T20:24:40.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.358+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431588 cookie 94609456635376 notifier 5615 bl.length()=238 2024-05-09T20:24:40.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.370+0000 7fce7ee3b980 10 cache get: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8 : miss 2024-05-09T20:24:40.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.370+0000 7fce7ee3b980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.370+0000 7fce7ee3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.370+0000 7fce7ee3b980 10 cache put: name=default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8 info.flags=0x0 2024-05-09T20:24:40.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.370+0000 7fce7ee3b980 10 adding default.rgw.log++pubsub..bucket.test-client.0-r-test-small-obj-data.3/6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8 to cache LRU end 2024-05-09T20:24:40.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.374+0000 7fce7ee3b980 10 removing key index: MAGDZXYRKRYDQIPQWLYT 2024-05-09T20:24:40.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.378+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:MAGDZXYRKRYDQIPQWLYT, ofs0, ns] 2024-05-09T20:24:40.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.378+0000 7fcd53fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431573 cookie 94609456639632 notifier 5615 bl.length()=170 2024-05-09T20:24:40.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.382+0000 7fce7ee3b980 10 removing email index: ragweed.client.0+test@test.test 2024-05-09T20:24:40.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.386+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.email:ragweed.client.0+test@test.test, ofs0, ns] 2024-05-09T20:24:40.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.386+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431622 cookie 94609456642448 notifier 5615 bl.length()=182 2024-05-09T20:24:40.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.390+0000 7fce7ee3b980 10 removing user buckets index 2024-05-09T20:24:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.394+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0.buckets, ofs0, ns] 2024-05-09T20:24:40.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.394+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431569 cookie 94609456650224 notifier 5615 bl.length()=173 2024-05-09T20:24:40.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.394+0000 7fce7ee3b980 10 removing user index: ragweed.client.0 2024-05-09T20:24:40.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.398+0000 7fce7ee3b980 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2024-05-09T20:24:40.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.398+0000 7fce7ee3b980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:ragweed.client.0, ofs0, ns] 2024-05-09T20:24:40.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.398+0000 7fcd537fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431589 cookie 94609456635376 notifier 5615 bl.length()=165 2024-05-09T20:24:40.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.410+0000 7fcd337e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:24:40.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.414+0000 7fce7ee3b980 20 remove_watcher() i=0 2024-05-09T20:24:40.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.414+0000 7fce7ee3b980 2 removed watcher, disabling cache 2024-05-09T20:24:40.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.418+0000 7fce7ee3b980 20 remove_watcher() i=1 2024-05-09T20:24:40.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.422+0000 7fce7ee3b980 20 remove_watcher() i=2 2024-05-09T20:24:40.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.422+0000 7fce7ee3b980 20 remove_watcher() i=3 2024-05-09T20:24:40.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.426+0000 7fce7ee3b980 20 remove_watcher() i=4 2024-05-09T20:24:40.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.430+0000 7fce7ee3b980 20 remove_watcher() i=5 2024-05-09T20:24:40.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.430+0000 7fce7ee3b980 20 remove_watcher() i=6 2024-05-09T20:24:40.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.434+0000 7fce7ee3b980 20 remove_watcher() i=7 2024-05-09T20:24:40.448 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2024-05-09T20:24:40.529 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:24:40.529 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:24:40.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 20 realm 2024-05-09T20:24:40.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:24:40.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.578+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.582+0000 7fa111351980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:24:40.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.582+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.582+0000 7fa111351980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:24:40.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.582+0000 7fa111351980 20 searching for the correct realm 2024-05-09T20:24:40.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.586+0000 7fa10f666640 0 -- 172.21.15.63:0/3329775300 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fa0f80d0070 msgr2=0x7fa0f80eb440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:24:40.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.590+0000 7fa107fff640 0 -- 172.21.15.63:0/3329775300 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fa100043040 msgr2=0x7fa100074a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:24:40.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:24:40.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:24:40.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:24:40.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:24:40.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:24:40.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:24:40.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 zone default found 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 4 Realm: () 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 zonegroup default 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.602+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.606+0000 7fa111351980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.606+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.606+0000 7fa111351980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:24:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.606+0000 7fa111351980 20 started sync module instance, tier type = 2024-05-09T20:24:40.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.606+0000 7fa111351980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:24:40.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 add_watcher() i=0 2024-05-09T20:24:40.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 add_watcher() i=1 2024-05-09T20:24:40.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 add_watcher() i=2 2024-05-09T20:24:40.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 add_watcher() i=3 2024-05-09T20:24:40.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 add_watcher() i=4 2024-05-09T20:24:40.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 add_watcher() i=5 2024-05-09T20:24:40.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 add_watcher() i=6 2024-05-09T20:24:40.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 add_watcher() i=7 2024-05-09T20:24:40.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 2 all 8 watchers are set, enabling cache 2024-05-09T20:24:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7f9fc57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:24:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:24:40.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.622+0000 7fa111351980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:24:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.702+0000 7f9f3ffff640 20 reqs_thread_entry: start 2024-05-09T20:24:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 20 init_complete bucket index max shards: 11 2024-05-09T20:24:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 20 Filter name: none 2024-05-09T20:24:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7f9f3dffb640 20 reqs_thread_entry: start 2024-05-09T20:24:40.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2024-05-09T20:24:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 20 rados->read ofs=0 len=0 2024-05-09T20:24:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 20 rados_obj.operate() r=0 bl.length=486 2024-05-09T20:24:40.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x13 2024-05-09T20:24:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2024-05-09T20:24:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:24:40.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.706+0000 7fa111351980 10 removing key index: WUUFATBQMBNHMZLDLEIV 2024-05-09T20:24:40.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.710+0000 7fa111351980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.keys:WUUFATBQMBNHMZLDLEIV, ofs0, ns] 2024-05-09T20:24:40.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.710+0000 7f9ff27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431570 cookie 94830612215216 notifier 5633 bl.length()=170 2024-05-09T20:24:40.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.710+0000 7fa111351980 10 removing email index: sysuser.client.0+test@test.test 2024-05-09T20:24:40.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.714+0000 7fa111351980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.email:sysuser.client.0+test@test.test, ofs0, ns] 2024-05-09T20:24:40.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.714+0000 7f9ff27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431590 cookie 94830612205072 notifier 5633 bl.length()=182 2024-05-09T20:24:40.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.718+0000 7fa111351980 10 removing user buckets index 2024-05-09T20:24:40.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.722+0000 7fa111351980 10 removing user index: sysuser.client.0 2024-05-09T20:24:40.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.722+0000 7fa111351980 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2024-05-09T20:24:40.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.722+0000 7fa111351980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:sysuser.client.0, ofs0, ns] 2024-05-09T20:24:40.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.722+0000 7f9ff1ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431623 cookie 94830612195328 notifier 5633 bl.length()=165 2024-05-09T20:24:40.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.726+0000 7f9fc57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:24:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.734+0000 7fa111351980 20 remove_watcher() i=0 2024-05-09T20:24:40.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.734+0000 7fa111351980 2 removed watcher, disabling cache 2024-05-09T20:24:40.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.734+0000 7fa111351980 20 remove_watcher() i=1 2024-05-09T20:24:40.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.738+0000 7fa111351980 20 remove_watcher() i=2 2024-05-09T20:24:40.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.738+0000 7fa111351980 20 remove_watcher() i=3 2024-05-09T20:24:40.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.742+0000 7fa111351980 20 remove_watcher() i=4 2024-05-09T20:24:40.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.742+0000 7fa111351980 20 remove_watcher() i=5 2024-05-09T20:24:40.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.746+0000 7fa111351980 20 remove_watcher() i=6 2024-05-09T20:24:40.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:24:40.746+0000 7fa111351980 20 remove_watcher() i=7 2024-05-09T20:24:40.756 INFO:tasks.ragweed:Removing ragweed... 2024-05-09T20:24:40.756 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/ragweed.client.0 2024-05-09T20:24:40.851 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T20:24:40.861 INFO:tasks.workunit:Pulling workunits from ref 86872c6b9201173c508ab909aaa65bdb12239dd6 2024-05-09T20:24:40.861 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T20:24:40.862 DEBUG:teuthology.orchestra.run.smithi063:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:24:40.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T20:24:40.868 INFO:teuthology.orchestra.run.smithi063.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-09T20:24:40.868 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:24:40.915 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-09T20:24:40.915 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-09T20:24:40.963 INFO:tasks.workunit:timeout=3h 2024-05-09T20:24:40.963 INFO:tasks.workunit:cleanup=True 2024-05-09T20:24:40.963 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/samarahu/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 86872c6b9201173c508ab909aaa65bdb12239dd6 2024-05-09T20:24:41.011 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T20:25:24.712 INFO:tasks.workunit.client.0.smithi063.stderr:Note: switching to '86872c6b9201173c508ab909aaa65bdb12239dd6'. 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:24.713 INFO:tasks.workunit.client.0.smithi063.stderr: git switch -c 2024-05-09T20:25:24.714 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:24.714 INFO:tasks.workunit.client.0.smithi063.stderr:Or undo this operation with: 2024-05-09T20:25:24.714 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:24.714 INFO:tasks.workunit.client.0.smithi063.stderr: git switch - 2024-05-09T20:25:24.714 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:24.714 INFO:tasks.workunit.client.0.smithi063.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T20:25:24.714 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:24.714 INFO:tasks.workunit.client.0.smithi063.stderr:HEAD is now at 86872c6b920 d4n: Add more in-depth config descriptions and improve naming 2024-05-09T20:25:24.717 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T20:25:24.768 INFO:tasks.workunit.client.0.smithi063.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T20:25:24.770 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T20:25:24.770 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T20:25:24.828 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T20:25:24.885 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T20:25:24.938 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T20:25:24.941 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T20:25:24.941 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T20:25:24.988 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T20:25:24.992 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:25:24.992 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T20:25:25.039 INFO:tasks.workunit:Running workunits matching rgw/run-reshard.sh on client.0... 2024-05-09T20:25:25.040 INFO:tasks.workunit:Running workunit rgw/run-reshard.sh... 2024-05-09T20:25:25.041 DEBUG:teuthology.orchestra.run.smithi063:workunit test rgw/run-reshard.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=86872c6b9201173c508ab909aaa65bdb12239dd6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-05-09T20:25:25.092 INFO:tasks.workunit.client.0.smithi063.stderr:+ ulimit -c 0 2024-05-09T20:25:25.092 INFO:tasks.workunit.client.0.smithi063.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/run-reshard.sh 2024-05-09T20:25:25.093 INFO:tasks.workunit.client.0.smithi063.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-09T20:25:25.093 INFO:tasks.workunit.client.0.smithi063.stderr:+ python3 -m venv /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-09T20:25:28.060 INFO:tasks.workunit.client.0.smithi063.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/activate 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ deactivate nondestructive 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n '' ']' 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n '' ']' 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ hash -r 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n '' ']' 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ unset VIRTUAL_ENV 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ unset VIRTUAL_ENV_PROMPT 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' '!' nondestructive = nondestructive ']' 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ VIRTUAL_ENV=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ export VIRTUAL_ENV 2024-05-09T20:25:28.061 INFO:tasks.workunit.client.0.smithi063.stderr:++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ PATH=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ export PATH 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n '' ']' 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -z '' ']' 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ _OLD_VIRTUAL_PS1= 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ PS1='(rgw) ' 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ export PS1 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ VIRTUAL_ENV_PROMPT='(rgw) ' 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ export VIRTUAL_ENV_PROMPT 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:++ hash -r 2024-05-09T20:25:28.062 INFO:tasks.workunit.client.0.smithi063.stderr:+ pip install pip --upgrade 2024-05-09T20:25:28.460 INFO:tasks.workunit.client.0.smithi063.stdout:Requirement already satisfied: pip in /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/lib/python3.10/site-packages (22.0.2) 2024-05-09T20:25:28.684 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting pip 2024-05-09T20:25:28.691 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached pip-24.0-py3-none-any.whl (2.1 MB) 2024-05-09T20:25:28.730 INFO:tasks.workunit.client.0.smithi063.stdout:Installing collected packages: pip 2024-05-09T20:25:28.730 INFO:tasks.workunit.client.0.smithi063.stdout: Attempting uninstall: pip 2024-05-09T20:25:28.730 INFO:tasks.workunit.client.0.smithi063.stdout: Found existing installation: pip 22.0.2 2024-05-09T20:25:29.006 INFO:tasks.workunit.client.0.smithi063.stdout: Uninstalling pip-22.0.2: 2024-05-09T20:25:29.012 INFO:tasks.workunit.client.0.smithi063.stdout: Successfully uninstalled pip-22.0.2 2024-05-09T20:25:29.976 INFO:tasks.workunit.client.0.smithi063.stdout:Successfully installed pip-24.0 2024-05-09T20:25:30.040 INFO:tasks.workunit.client.0.smithi063.stderr:+ pip install boto3 2024-05-09T20:25:31.018 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting boto3 2024-05-09T20:25:31.020 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached boto3-1.34.101-py3-none-any.whl.metadata (6.6 kB) 2024-05-09T20:25:31.590 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting botocore<1.35.0,>=1.34.101 (from boto3) 2024-05-09T20:25:31.592 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached botocore-1.34.101-py3-none-any.whl.metadata (5.7 kB) 2024-05-09T20:25:31.647 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting jmespath<2.0.0,>=0.7.1 (from boto3) 2024-05-09T20:25:31.648 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached jmespath-1.0.1-py3-none-any.whl.metadata (7.6 kB) 2024-05-09T20:25:31.703 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting s3transfer<0.11.0,>=0.10.0 (from boto3) 2024-05-09T20:25:31.704 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached s3transfer-0.10.1-py3-none-any.whl.metadata (1.7 kB) 2024-05-09T20:25:31.776 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting python-dateutil<3.0.0,>=2.1 (from botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:25:31.777 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB) 2024-05-09T20:25:31.879 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting urllib3!=2.2.0,<3,>=1.25.4 (from botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:25:31.881 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached urllib3-2.2.1-py3-none-any.whl.metadata (6.4 kB) 2024-05-09T20:25:32.059 INFO:tasks.workunit.client.0.smithi063.stdout:Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.101->boto3) 2024-05-09T20:25:32.060 INFO:tasks.workunit.client.0.smithi063.stdout: Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB) 2024-05-09T20:25:32.071 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached boto3-1.34.101-py3-none-any.whl (139 kB) 2024-05-09T20:25:32.073 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached botocore-1.34.101-py3-none-any.whl (12.2 MB) 2024-05-09T20:25:32.094 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached jmespath-1.0.1-py3-none-any.whl (20 kB) 2024-05-09T20:25:32.095 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB) 2024-05-09T20:25:32.097 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 2024-05-09T20:25:32.098 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached urllib3-2.2.1-py3-none-any.whl (121 kB) 2024-05-09T20:25:32.100 INFO:tasks.workunit.client.0.smithi063.stdout:Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) 2024-05-09T20:25:32.195 INFO:tasks.workunit.client.0.smithi063.stdout:Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3 2024-05-09T20:25:33.453 INFO:tasks.workunit.client.0.smithi063.stdout:Successfully installed boto3-1.34.101 botocore-1.34.101 jmespath-1.0.1 python-dateutil-2.9.0.post0 s3transfer-0.10.1 six-1.16.0 urllib3-2.2.1 2024-05-09T20:25:33.618 INFO:tasks.workunit.client.0.smithi063.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/bin/python3 /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_reshard.py 2024-05-09T20:25:33.763 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin user create --uid tester --display-name "Testing" --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn 2024-05-09T20:25:33.802 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:33.802 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:33.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:33.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:33.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 realm 2024-05-09T20:25:33.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:33.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:33.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:33.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:33.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:33.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:33.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:33.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.857+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:33.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.861+0000 7f2911e29980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:33.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.861+0000 7f2911e29980 20 searching for the correct realm 2024-05-09T20:25:33.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:33.861+0000 7f290f13c640 0 -- 172.21.15.63:0/968643672 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f2900004040 msgr2=0x7f29000b7920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:34.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.073+0000 7f290f13c640 0 -- 172.21.15.63:0/968643672 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f29000f1400 msgr2=0x7f29000f1860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:34.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:34.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:34.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:34.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:34.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:34.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:34.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:34.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:34.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:34.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:34.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:34.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:34.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 zone default found 2024-05-09T20:25:34.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 4 Realm: () 2024-05-09T20:25:34.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:34.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:34.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:34.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 zonegroup default 2024-05-09T20:25:34.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.297+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:34.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.301+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:34.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.301+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:34.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.301+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:34.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.301+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:34.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.301+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:34.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.301+0000 7f2911e29980 20 started sync module instance, tier type = 2024-05-09T20:25:34.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.301+0000 7f2911e29980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:34.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.325+0000 7f2911e29980 20 add_watcher() i=0 2024-05-09T20:25:34.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.325+0000 7f2911e29980 20 add_watcher() i=1 2024-05-09T20:25:34.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.325+0000 7f2911e29980 20 add_watcher() i=2 2024-05-09T20:25:34.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.325+0000 7f2911e29980 20 add_watcher() i=3 2024-05-09T20:25:34.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.325+0000 7f2911e29980 20 add_watcher() i=4 2024-05-09T20:25:34.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.325+0000 7f2911e29980 20 add_watcher() i=5 2024-05-09T20:25:34.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.325+0000 7f2911e29980 20 add_watcher() i=6 2024-05-09T20:25:34.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.329+0000 7f2911e29980 20 add_watcher() i=7 2024-05-09T20:25:34.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.329+0000 7f2911e29980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:34.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.329+0000 7f27c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:34.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:34.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:34.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 5 note: GC not initialized 2024-05-09T20:25:34.322 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f27c4fe1640 20 reqs_thread_entry: start 2024-05-09T20:25:34.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:34.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 20 Filter name: none 2024-05-09T20:25:34.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f273e7fc640 20 reqs_thread_entry: start 2024-05-09T20:25:34.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 10 cache get: name=default.rgw.meta+users.uid+tester : miss 2024-05-09T20:25:34.323 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x0 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 10 adding default.rgw.meta+users.uid+tester to cache LRU end 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 20 rados->read ofs=0 len=0 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-05-09T20:25:34.324 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.333+0000 7f2911e29980 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-05-09T20:25:34.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.337+0000 7f2911e29980 10 cache put: name=default.rgw.meta+users.uid+tester info.flags=0x17 2024-05-09T20:25:34.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.337+0000 7f2911e29980 10 moving default.rgw.meta+users.uid+tester to cache LRU end 2024-05-09T20:25:34.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.337+0000 7f2911e29980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester, ofs0, ns] 2024-05-09T20:25:34.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.337+0000 7f27eeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431571 cookie 94589758864912 notifier 5648 bl.length()=569 2024-05-09T20:25:34.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.337+0000 7f27eeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester info.flags=0x17 2024-05-09T20:25:34.327 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.337+0000 7f27eeffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester to cache LRU end 2024-05-09T20:25:34.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.341+0000 7f2911e29980 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-05-09T20:25:34.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.341+0000 7f2911e29980 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-09T20:25:34.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.341+0000 7f2911e29980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-05-09T20:25:34.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.341+0000 7f27eeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431572 cookie 94589758863792 notifier 5648 bl.length()=180 2024-05-09T20:25:34.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.341+0000 7f27eeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-05-09T20:25:34.332 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.341+0000 7f27eeffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-09T20:25:34.335 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.345+0000 7f27c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:34.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.349+0000 7f2911e29980 20 remove_watcher() i=0 2024-05-09T20:25:34.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.349+0000 7f2911e29980 2 removed watcher, disabling cache 2024-05-09T20:25:34.342 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.353+0000 7f2911e29980 20 remove_watcher() i=1 2024-05-09T20:25:34.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.353+0000 7f2911e29980 20 remove_watcher() i=2 2024-05-09T20:25:34.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.357+0000 7f2911e29980 20 remove_watcher() i=3 2024-05-09T20:25:34.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.357+0000 7f2911e29980 20 remove_watcher() i=4 2024-05-09T20:25:34.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.361+0000 7f2911e29980 20 remove_watcher() i=5 2024-05-09T20:25:34.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.361+0000 7f2911e29980 20 remove_watcher() i=6 2024-05-09T20:25:34.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:34.361+0000 7f2911e29980 20 remove_watcher() i=7 2024-05-09T20:25:35.034 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard add 2024-05-09T20:25:35.034 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:35.035 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-09T20:25:35.069 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:35.069 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:35.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.125+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 realm 2024-05-09T20:25:35.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:35.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:35.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:35.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.129+0000 7f49e19ef980 20 searching for the correct realm 2024-05-09T20:25:35.124 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.133+0000 7f49dfd04640 0 -- 172.21.15.63:0/2262123463 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f49d4008040 msgr2=0x7f49d403d400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:35.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.345+0000 7f49ded02640 0 -- 172.21.15.63:0/2262123463 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f49d00f06c0 msgr2=0x7f49d00c1940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:35.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.361+0000 7f49e19ef980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:35.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.361+0000 7f49e19ef980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:35.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.361+0000 7f49e19ef980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:35.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.361+0000 7f49e19ef980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:35.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.361+0000 7f49e19ef980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:35.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.361+0000 7f49e19ef980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:35.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.361+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:35.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 zone default found 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 4 Realm: () 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 zonegroup default 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.365+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.369+0000 7f49e19ef980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.369+0000 7f49e19ef980 20 started sync module instance, tier type = 2024-05-09T20:25:35.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.369+0000 7f49e19ef980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:35.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.389+0000 7f49e19ef980 20 add_watcher() i=0 2024-05-09T20:25:35.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.389+0000 7f49e19ef980 20 add_watcher() i=1 2024-05-09T20:25:35.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.389+0000 7f49e19ef980 20 add_watcher() i=2 2024-05-09T20:25:35.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.389+0000 7f49e19ef980 20 add_watcher() i=3 2024-05-09T20:25:35.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.389+0000 7f49e19ef980 20 add_watcher() i=4 2024-05-09T20:25:35.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.389+0000 7f49e19ef980 20 add_watcher() i=5 2024-05-09T20:25:35.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.389+0000 7f49e19ef980 20 add_watcher() i=6 2024-05-09T20:25:35.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.393+0000 7f49e19ef980 20 add_watcher() i=7 2024-05-09T20:25:35.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.393+0000 7f49e19ef980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:35.385 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.393+0000 7f4895fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:35.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:35.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:35.386 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 5 note: GC not initialized 2024-05-09T20:25:35.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f4894fe1640 20 reqs_thread_entry: start 2024-05-09T20:25:35.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:35.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 Filter name: none 2024-05-09T20:25:35.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f480e7fc640 20 reqs_thread_entry: start 2024-05-09T20:25:35.387 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:35.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:35.388 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:35.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.397+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.401+0000 7f49e19ef980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:35.389 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.401+0000 7f49e19ef980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.390 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.401+0000 7f49e19ef980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:35.390 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.401+0000 7f49e19ef980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:35.393 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.401+0000 7f4895fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:35.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.409+0000 7f49e19ef980 20 remove_watcher() i=0 2024-05-09T20:25:35.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.409+0000 7f49e19ef980 2 removed watcher, disabling cache 2024-05-09T20:25:35.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.413+0000 7f49e19ef980 20 remove_watcher() i=1 2024-05-09T20:25:35.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.413+0000 7f49e19ef980 20 remove_watcher() i=2 2024-05-09T20:25:35.406 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.417+0000 7f49e19ef980 20 remove_watcher() i=3 2024-05-09T20:25:35.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.417+0000 7f49e19ef980 20 remove_watcher() i=4 2024-05-09T20:25:35.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.421+0000 7f49e19ef980 20 remove_watcher() i=5 2024-05-09T20:25:35.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.421+0000 7f49e19ef980 20 remove_watcher() i=6 2024-05-09T20:25:35.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.421+0000 7f49e19ef980 20 remove_watcher() i=7 2024-05-09T20:25:35.428 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:35.428 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:35.429 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: a-bucket id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 num_objects: 0 size_kb: 0 num_shards: 11 2024-05-09T20:25:35.429 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:35.429 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 12 2024-05-09T20:25:35.458 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:35.458 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:35.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 20 realm 2024-05-09T20:25:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.513+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.517+0000 7fd12aebf980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:35.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.517+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.517+0000 7fd12aebf980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:35.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.517+0000 7fd12aebf980 20 searching for the correct realm 2024-05-09T20:25:35.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.521+0000 7fd1291d4640 0 -- 172.21.15.63:0/3431086636 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fd11c0037c0 msgr2=0x7fd11c0cf440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:35.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.529+0000 7fd1281d2640 0 -- 172.21.15.63:0/3431086636 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fd118018070 msgr2=0x7fd118054170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:25:35.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:35.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:35.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:35.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:35.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:35.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:35.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:35.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:35.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 zone default found 2024-05-09T20:25:35.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 4 Realm: () 2024-05-09T20:25:35.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:35.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:35.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:35.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 zonegroup default 2024-05-09T20:25:35.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 started sync module instance, tier type = 2024-05-09T20:25:35.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.541+0000 7fd12aebf980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:35.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.557+0000 7fd12aebf980 20 add_watcher() i=0 2024-05-09T20:25:35.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.557+0000 7fd12aebf980 20 add_watcher() i=1 2024-05-09T20:25:35.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.557+0000 7fd12aebf980 20 add_watcher() i=2 2024-05-09T20:25:35.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.557+0000 7fd12aebf980 20 add_watcher() i=3 2024-05-09T20:25:35.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.557+0000 7fd12aebf980 20 add_watcher() i=4 2024-05-09T20:25:35.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.557+0000 7fd12aebf980 20 add_watcher() i=5 2024-05-09T20:25:35.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.557+0000 7fd12aebf980 20 add_watcher() i=6 2024-05-09T20:25:35.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 20 add_watcher() i=7 2024-05-09T20:25:35.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:35.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fcfdefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:35.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:35.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:35.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 5 note: GC not initialized 2024-05-09T20:25:35.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fcfddfe3640 20 reqs_thread_entry: start 2024-05-09T20:25:35.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:35.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 20 Filter name: none 2024-05-09T20:25:35.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fcf4f7fe640 20 reqs_thread_entry: start 2024-05-09T20:25:35.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-09T20:25:35.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.561+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:35.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-05-09T20:25:35.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-09T20:25:35.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 : miss 2024-05-09T20:25:35.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:35.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:35.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:35.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:35.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:35.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.565+0000 7fd12aebf980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 2024-05-09T20:25:35.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.569+0000 7fcfdefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:35.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.573+0000 7fd12aebf980 20 remove_watcher() i=0 2024-05-09T20:25:35.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.573+0000 7fd12aebf980 2 removed watcher, disabling cache 2024-05-09T20:25:35.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.573+0000 7fd12aebf980 20 remove_watcher() i=1 2024-05-09T20:25:35.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.577+0000 7fd12aebf980 20 remove_watcher() i=2 2024-05-09T20:25:35.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.577+0000 7fd12aebf980 20 remove_watcher() i=3 2024-05-09T20:25:35.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.581+0000 7fd12aebf980 20 remove_watcher() i=4 2024-05-09T20:25:35.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.581+0000 7fd12aebf980 20 remove_watcher() i=5 2024-05-09T20:25:35.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.585+0000 7fd12aebf980 20 remove_watcher() i=6 2024-05-09T20:25:35.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.585+0000 7fd12aebf980 20 remove_watcher() i=7 2024-05-09T20:25:35.589 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin reshard list 2024-05-09T20:25:35.613 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:35.613 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.665+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 realm 2024-05-09T20:25:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:35.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.669+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.673+0000 7fa706283980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:35.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.673+0000 7fa706283980 20 searching for the correct realm 2024-05-09T20:25:35.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.677+0000 7fa703d97640 0 -- 172.21.15.63:0/3687377682 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fa6fc044070 msgr2=0x7fa6fc036450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:35.884 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.893+0000 7fa703d97640 0 -- 172.21.15.63:0/3687377682 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fa6fc044070 msgr2=0x7fa6fc036450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:35.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:35.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:35.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:35.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:35.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:35.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:35.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.897 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.905+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:35.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:35.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 zone default found 2024-05-09T20:25:35.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 4 Realm: () 2024-05-09T20:25:35.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:35.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:35.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:35.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 zonegroup default 2024-05-09T20:25:35.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados->read ofs=0 len=0 2024-05-09T20:25:35.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:35.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 started sync module instance, tier type = 2024-05-09T20:25:35.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.909+0000 7fa706283980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:35.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 20 add_watcher() i=0 2024-05-09T20:25:35.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 20 add_watcher() i=1 2024-05-09T20:25:35.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 20 add_watcher() i=2 2024-05-09T20:25:35.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 20 add_watcher() i=3 2024-05-09T20:25:35.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 20 add_watcher() i=4 2024-05-09T20:25:35.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 20 add_watcher() i=5 2024-05-09T20:25:35.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 20 add_watcher() i=6 2024-05-09T20:25:35.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 20 add_watcher() i=7 2024-05-09T20:25:35.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.937+0000 7fa706283980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:35.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.941+0000 7fa5b9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:35.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.941+0000 7fa706283980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:35.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.941+0000 7fa706283980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:35.933 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.941+0000 7fa706283980 5 note: GC not initialized 2024-05-09T20:25:35.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.945+0000 7fa5b8fe1640 20 reqs_thread_entry: start 2024-05-09T20:25:35.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.945+0000 7fa706283980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:35.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.945+0000 7fa706283980 20 Filter name: none 2024-05-09T20:25:35.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.945+0000 7fa5327fc640 20 reqs_thread_entry: start 2024-05-09T20:25:35.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.957+0000 7fa5b9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:35.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.961+0000 7fa706283980 20 remove_watcher() i=0 2024-05-09T20:25:35.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.961+0000 7fa706283980 2 removed watcher, disabling cache 2024-05-09T20:25:35.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.965+0000 7fa706283980 20 remove_watcher() i=1 2024-05-09T20:25:35.957 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.965+0000 7fa706283980 20 remove_watcher() i=2 2024-05-09T20:25:35.960 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.969+0000 7fa706283980 20 remove_watcher() i=3 2024-05-09T20:25:35.962 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.973+0000 7fa706283980 20 remove_watcher() i=4 2024-05-09T20:25:35.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.973+0000 7fa706283980 20 remove_watcher() i=5 2024-05-09T20:25:35.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.977+0000 7fa706283980 20 remove_watcher() i=6 2024-05-09T20:25:35.968 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:35.977+0000 7fa706283980 20 remove_watcher() i=7 2024-05-09T20:25:35.984 INFO:tasks.workunit.client.0.smithi063.stderr:bucket name a-bucket 2024-05-09T20:25:35.984 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard process 2024-05-09T20:25:35.984 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:35.984 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin reshard process 2024-05-09T20:25:36.013 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:36.014 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:36.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:36.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 20 realm 2024-05-09T20:25:36.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:36.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:36.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:36.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.069+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.073+0000 7f433b5e5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:36.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.073+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.073+0000 7f433b5e5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:36.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.073+0000 7f433b5e5980 20 searching for the correct realm 2024-05-09T20:25:36.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.077+0000 7f43398fa640 0 -- 172.21.15.63:0/3884718321 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f433001a040 msgr2=0x7f433008b870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:36.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.081+0000 7f43388f8640 0 -- 172.21.15.63:0/3884718321 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f432c0dc070 msgr2=0x7f432c1172a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:36.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.089+0000 7f433b5e5980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:36.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.089+0000 7f433b5e5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:36.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.089+0000 7f433b5e5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:36.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.089+0000 7f433b5e5980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:36.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.089+0000 7f433b5e5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:36.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.089+0000 7f433b5e5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:36.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.089+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:36.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 zone default found 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 4 Realm: () 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:36.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 zonegroup default 2024-05-09T20:25:36.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:36.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:36.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:36.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 started sync module instance, tier type = 2024-05-09T20:25:36.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.093+0000 7f433b5e5980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:36.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 20 add_watcher() i=0 2024-05-09T20:25:36.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 20 add_watcher() i=1 2024-05-09T20:25:36.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 20 add_watcher() i=2 2024-05-09T20:25:36.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 20 add_watcher() i=3 2024-05-09T20:25:36.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 20 add_watcher() i=4 2024-05-09T20:25:36.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 20 add_watcher() i=5 2024-05-09T20:25:36.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 20 add_watcher() i=6 2024-05-09T20:25:36.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 20 add_watcher() i=7 2024-05-09T20:25:36.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.109+0000 7f433b5e5980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:36.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f41ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:36.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f433b5e5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:36.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f433b5e5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:36.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f433b5e5980 5 note: GC not initialized 2024-05-09T20:25:36.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f41ee7e4640 20 reqs_thread_entry: start 2024-05-09T20:25:36.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f433b5e5980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:36.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f433b5e5980 20 Filter name: none 2024-05-09T20:25:36.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f415ffff640 20 reqs_thread_entry: start 2024-05-09T20:25:36.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.113+0000 7f433b5e5980 20 processing logshard = reshard.0000000000 2024-05-09T20:25:36.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.117+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-05-09T20:25:36.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.117+0000 7f433b5e5980 20 processing logshard = reshard.0000000001 2024-05-09T20:25:36.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.121+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-05-09T20:25:36.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.121+0000 7f433b5e5980 20 processing logshard = reshard.0000000002 2024-05-09T20:25:36.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.129+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-05-09T20:25:36.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.129+0000 7f433b5e5980 20 processing logshard = reshard.0000000003 2024-05-09T20:25:36.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.133+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-05-09T20:25:36.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.133+0000 7f433b5e5980 20 processing logshard = reshard.0000000004 2024-05-09T20:25:36.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.141+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-05-09T20:25:36.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.141+0000 7f433b5e5980 20 processing logshard = reshard.0000000005 2024-05-09T20:25:36.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-05-09T20:25:36.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 20 processing logshard = reshard.0000000006 2024-05-09T20:25:36.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 20 process_entry resharding a-bucket 2024-05-09T20:25:36.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-09T20:25:36.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:36.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-05-09T20:25:36.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-09T20:25:36.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 : miss 2024-05-09T20:25:36.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.145+0000 7f433b5e5980 20 rados->read ofs=0 len=0 2024-05-09T20:25:36.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.149+0000 7f433b5e5980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:36.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.149+0000 7f433b5e5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:36.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.149+0000 7f433b5e5980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:36.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.149+0000 7f433b5e5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:36.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.149+0000 7f433b5e5980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:36.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.149+0000 7f433b5e5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 2024-05-09T20:25:36.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.161+0000 7f433b5e5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:36.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.161+0000 7f433b5e5980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:36.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.161+0000 7f433b5e5980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:36.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.161+0000 7f433b5e5980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17, ofs0, ns] 2024-05-09T20:25:36.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.161+0000 7f421cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431576 cookie 94074277380800 notifier 6039 bl.length()=819 2024-05-09T20:25:36.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.161+0000 7f421cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:36.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.161+0000 7f421cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:36.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.161+0000 7f421cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:36.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.165+0000 7f433b5e5980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_sources= new_sources= 2024-05-09T20:25:36.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.165+0000 7f433b5e5980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential sources added=[] removed=[] 2024-05-09T20:25:36.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.165+0000 7f433b5e5980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_dests= new_dests= 2024-05-09T20:25:36.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.165+0000 7f433b5e5980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential dests added=[] removed=[] 2024-05-09T20:25:36.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.0 2024-05-09T20:25:36.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.0 2024-05-09T20:25:36.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.1 2024-05-09T20:25:36.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.1 2024-05-09T20:25:36.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.2 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.2 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.3 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.3 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.4 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.4 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.5 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.5 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.6 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.6 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.7 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.7 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.8 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.8 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.9 2024-05-09T20:25:36.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.9 2024-05-09T20:25:36.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.10 2024-05-09T20:25:36.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.10 2024-05-09T20:25:36.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.11 2024-05-09T20:25:36.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.11 2024-05-09T20:25:36.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.0 2024-05-09T20:25:36.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1 2024-05-09T20:25:36.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.169+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2 2024-05-09T20:25:36.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.173+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.3 2024-05-09T20:25:36.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.173+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.4 2024-05-09T20:25:36.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.173+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.5 2024-05-09T20:25:36.164 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.173+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.6 2024-05-09T20:25:36.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.173+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.7 2024-05-09T20:25:36.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.177+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.8 2024-05-09T20:25:36.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.177+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.9 2024-05-09T20:25:36.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.177+0000 7f433b5e5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.10 2024-05-09T20:25:36.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.177+0000 7f433b5e5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 : hit (requested=0x17, cached=0x17) 2024-05-09T20:25:36.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.177+0000 7f433b5e5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 2024-05-09T20:25:36.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.181+0000 7f433b5e5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:36.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.181+0000 7f433b5e5980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:36.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.181+0000 7f433b5e5980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:36.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.181+0000 7f433b5e5980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17, ofs0, ns] 2024-05-09T20:25:36.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.181+0000 7f421cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431577 cookie 94074277380800 notifier 6039 bl.length()=787 2024-05-09T20:25:36.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.181+0000 7f421cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:36.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.181+0000 7f421cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:36.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.181+0000 7f421cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:36.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.185+0000 7f433b5e5980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_sources= new_sources= 2024-05-09T20:25:36.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.185+0000 7f433b5e5980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential sources added=[] removed=[] 2024-05-09T20:25:36.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.185+0000 7f433b5e5980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_dests= new_dests= 2024-05-09T20:25:36.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.185+0000 7f433b5e5980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential dests added=[] removed=[] 2024-05-09T20:25:36.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.189+0000 7f433b5e5980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-05-09T20:25:36.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.189+0000 7f433b5e5980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-05-09T20:25:36.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.193+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-05-09T20:25:36.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.193+0000 7f433b5e5980 20 processing logshard = reshard.0000000007 2024-05-09T20:25:36.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.201+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-05-09T20:25:36.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.201+0000 7f433b5e5980 20 processing logshard = reshard.0000000008 2024-05-09T20:25:36.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.205+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-05-09T20:25:36.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.205+0000 7f433b5e5980 20 processing logshard = reshard.0000000009 2024-05-09T20:25:36.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.213+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-05-09T20:25:36.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.213+0000 7f433b5e5980 20 processing logshard = reshard.0000000010 2024-05-09T20:25:36.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.217+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-05-09T20:25:36.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.217+0000 7f433b5e5980 20 processing logshard = reshard.0000000011 2024-05-09T20:25:36.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.221+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-05-09T20:25:36.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.221+0000 7f433b5e5980 20 processing logshard = reshard.0000000012 2024-05-09T20:25:36.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.225+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-05-09T20:25:36.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.225+0000 7f433b5e5980 20 processing logshard = reshard.0000000013 2024-05-09T20:25:36.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.233+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-05-09T20:25:36.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.233+0000 7f433b5e5980 20 processing logshard = reshard.0000000014 2024-05-09T20:25:36.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.237+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-05-09T20:25:36.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.237+0000 7f433b5e5980 20 processing logshard = reshard.0000000015 2024-05-09T20:25:36.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.241+0000 7f433b5e5980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-05-09T20:25:36.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.245+0000 7f41ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:36.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.249+0000 7f433b5e5980 20 remove_watcher() i=0 2024-05-09T20:25:36.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.249+0000 7f433b5e5980 2 removed watcher, disabling cache 2024-05-09T20:25:36.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.253+0000 7f433b5e5980 20 remove_watcher() i=1 2024-05-09T20:25:36.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.257+0000 7f433b5e5980 20 remove_watcher() i=2 2024-05-09T20:25:36.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.257+0000 7f433b5e5980 20 remove_watcher() i=3 2024-05-09T20:25:36.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.261+0000 7f433b5e5980 20 remove_watcher() i=4 2024-05-09T20:25:36.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.261+0000 7f433b5e5980 20 remove_watcher() i=5 2024-05-09T20:25:36.254 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.265+0000 7f433b5e5980 20 remove_watcher() i=6 2024-05-09T20:25:36.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:36.265+0000 7f433b5e5980 20 remove_watcher() i=7 2024-05-09T20:25:41.279 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-09T20:25:41.313 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:41.313 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:41.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.369+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:41.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 realm 2024-05-09T20:25:41.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:41.362 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:41.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:41.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.364 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:41.364 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:41.365 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.373+0000 7f9bd9883980 20 searching for the correct realm 2024-05-09T20:25:41.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.377+0000 7f9bd7397640 0 -- 172.21.15.63:0/3828926755 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f9bd0084840 msgr2=0x7f9bd006a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:41.584 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.593+0000 7f9bd7397640 0 -- 172.21.15.63:0/3828926755 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f9bd0084840 msgr2=0x7f9bd006a7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:41.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:41.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:41.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:41.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:41.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:41.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:41.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:41.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:41.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.605+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:41.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 zone default found 2024-05-09T20:25:41.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 4 Realm: () 2024-05-09T20:25:41.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:41.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:41.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:41.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 zonegroup default 2024-05-09T20:25:41.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:41.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:41.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:41.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 started sync module instance, tier type = 2024-05-09T20:25:41.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.609+0000 7f9bd9883980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:41.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 20 add_watcher() i=0 2024-05-09T20:25:41.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 20 add_watcher() i=1 2024-05-09T20:25:41.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 20 add_watcher() i=2 2024-05-09T20:25:41.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 20 add_watcher() i=3 2024-05-09T20:25:41.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 20 add_watcher() i=4 2024-05-09T20:25:41.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 20 add_watcher() i=5 2024-05-09T20:25:41.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 20 add_watcher() i=6 2024-05-09T20:25:41.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 20 add_watcher() i=7 2024-05-09T20:25:41.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.633+0000 7f9bd9883980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:41.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.637+0000 7f9a8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:41.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9bd9883980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:41.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9bd9883980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:41.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9bd9883980 5 note: GC not initialized 2024-05-09T20:25:41.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9a8cfe1640 20 reqs_thread_entry: start 2024-05-09T20:25:41.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9bd9883980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:41.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9bd9883980 20 Filter name: none 2024-05-09T20:25:41.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9a067fc640 20 reqs_thread_entry: start 2024-05-09T20:25:41.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9bd9883980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:41.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.641+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.645+0000 7f9bd9883980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:41.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.645+0000 7f9bd9883980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:41.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.645+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.645+0000 7f9bd9883980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:41.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.645+0000 7f9bd9883980 20 rados->read ofs=0 len=0 2024-05-09T20:25:41.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.645+0000 7f9bd9883980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:41.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.645+0000 7f9bd9883980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:41.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.649+0000 7f9a8dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:41.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.653+0000 7f9bd9883980 20 remove_watcher() i=0 2024-05-09T20:25:41.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.653+0000 7f9bd9883980 2 removed watcher, disabling cache 2024-05-09T20:25:41.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.657+0000 7f9bd9883980 20 remove_watcher() i=1 2024-05-09T20:25:41.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.657+0000 7f9bd9883980 20 remove_watcher() i=2 2024-05-09T20:25:41.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.661+0000 7f9bd9883980 20 remove_watcher() i=3 2024-05-09T20:25:41.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.661+0000 7f9bd9883980 20 remove_watcher() i=4 2024-05-09T20:25:41.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.665+0000 7f9bd9883980 20 remove_watcher() i=5 2024-05-09T20:25:41.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.665+0000 7f9bd9883980 20 remove_watcher() i=6 2024-05-09T20:25:41.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:41.665+0000 7f9bd9883980 20 remove_watcher() i=7 2024-05-09T20:25:41.674 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:41.674 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:41.674 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: a-bucket id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 num_objects: 0 size_kb: 0 num_shards: 12 2024-05-09T20:25:41.674 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:41.674 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard add non empty bucket 2024-05-09T20:25:41.674 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:42.394 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-09T20:25:42.428 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:42.428 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:42.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 20 realm 2024-05-09T20:25:42.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:42.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.485+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.489+0000 7ff1cdda3980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:42.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.489+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.489+0000 7ff1cdda3980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:42.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.489+0000 7ff1cdda3980 20 searching for the correct realm 2024-05-09T20:25:42.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.493+0000 7ff1cb8b7640 0 -- 172.21.15.63:0/2371522473 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7ff1c4003bb0 msgr2=0x7ff1c4008ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:42.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.709+0000 7ff1cb8b7640 0 -- 172.21.15.63:0/2371522473 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7ff1c4003bb0 msgr2=0x7ff1c4008ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:42.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:42.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:42.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:42.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:42.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:42.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:42.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:42.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:42.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 zone default found 2024-05-09T20:25:42.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 4 Realm: () 2024-05-09T20:25:42.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:42.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:42.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:42.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 zonegroup default 2024-05-09T20:25:42.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.721+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.725+0000 7ff1cdda3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.725+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.725+0000 7ff1cdda3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.725+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.725+0000 7ff1cdda3980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.725+0000 7ff1cdda3980 20 started sync module instance, tier type = 2024-05-09T20:25:42.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.725+0000 7ff1cdda3980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:42.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.749+0000 7ff1cdda3980 20 add_watcher() i=0 2024-05-09T20:25:42.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.749+0000 7ff1cdda3980 20 add_watcher() i=1 2024-05-09T20:25:42.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.749+0000 7ff1cdda3980 20 add_watcher() i=2 2024-05-09T20:25:42.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.749+0000 7ff1cdda3980 20 add_watcher() i=3 2024-05-09T20:25:42.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.749+0000 7ff1cdda3980 20 add_watcher() i=4 2024-05-09T20:25:42.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.749+0000 7ff1cdda3980 20 add_watcher() i=5 2024-05-09T20:25:42.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.749+0000 7ff1cdda3980 20 add_watcher() i=6 2024-05-09T20:25:42.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.753+0000 7ff1cdda3980 20 add_watcher() i=7 2024-05-09T20:25:42.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.753+0000 7ff1cdda3980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:42.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff081fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:42.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff1cdda3980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:42.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff1cdda3980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:42.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff1cdda3980 5 note: GC not initialized 2024-05-09T20:25:42.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff080fe1640 20 reqs_thread_entry: start 2024-05-09T20:25:42.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff1cdda3980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:42.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff1cdda3980 20 Filter name: none 2024-05-09T20:25:42.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7feffa7fc640 20 reqs_thread_entry: start 2024-05-09T20:25:42.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff1cdda3980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:42.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.757+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.761+0000 7ff1cdda3980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:42.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.761+0000 7ff1cdda3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:42.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.761+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.761+0000 7ff1cdda3980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:42.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.761+0000 7ff1cdda3980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.761+0000 7ff1cdda3980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:42.751 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.761+0000 7ff1cdda3980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:42.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.765+0000 7ff081fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:42.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.769+0000 7ff1cdda3980 20 remove_watcher() i=0 2024-05-09T20:25:42.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.769+0000 7ff1cdda3980 2 removed watcher, disabling cache 2024-05-09T20:25:42.761 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.769+0000 7ff1cdda3980 20 remove_watcher() i=1 2024-05-09T20:25:42.763 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.773+0000 7ff1cdda3980 20 remove_watcher() i=2 2024-05-09T20:25:42.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.777+0000 7ff1cdda3980 20 remove_watcher() i=3 2024-05-09T20:25:42.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.777+0000 7ff1cdda3980 20 remove_watcher() i=4 2024-05-09T20:25:42.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.777+0000 7ff1cdda3980 20 remove_watcher() i=5 2024-05-09T20:25:42.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.781+0000 7ff1cdda3980 20 remove_watcher() i=6 2024-05-09T20:25:42.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.781+0000 7ff1cdda3980 20 remove_watcher() i=7 2024-05-09T20:25:42.788 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:42.788 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:42.789 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: a-bucket id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 num_objects: 8 size_kb: 1 num_shards: 12 2024-05-09T20:25:42.789 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:42.789 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin reshard add --bucket a-bucket --num-shards 13 2024-05-09T20:25:42.818 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:42.819 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:42.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 20 realm 2024-05-09T20:25:42.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:42.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:42.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.873+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.877+0000 7fad07aad980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:42.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.877+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:42.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.877+0000 7fad07aad980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:42.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.877+0000 7fad07aad980 20 searching for the correct realm 2024-05-09T20:25:42.871 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:42.881+0000 7fad05dc2640 0 -- 172.21.15.63:0/1217617262 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7facfc127040 msgr2=0x7facfc0d06f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:43.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.093+0000 7fad04dc0640 0 -- 172.21.15.63:0/1217617262 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7facf8027020 msgr2=0x7facf8056010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:43.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:43.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:43.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:43.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:43.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:43.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:43.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.109+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:43.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:43.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 zone default found 2024-05-09T20:25:43.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 4 Realm: () 2024-05-09T20:25:43.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:43.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:43.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:43.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 zonegroup default 2024-05-09T20:25:43.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 started sync module instance, tier type = 2024-05-09T20:25:43.104 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.113+0000 7fad07aad980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:43.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 20 add_watcher() i=0 2024-05-09T20:25:43.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 20 add_watcher() i=1 2024-05-09T20:25:43.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 20 add_watcher() i=2 2024-05-09T20:25:43.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 20 add_watcher() i=3 2024-05-09T20:25:43.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 20 add_watcher() i=4 2024-05-09T20:25:43.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 20 add_watcher() i=5 2024-05-09T20:25:43.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 20 add_watcher() i=6 2024-05-09T20:25:43.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 20 add_watcher() i=7 2024-05-09T20:25:43.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.141+0000 7fad07aad980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:43.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.145+0000 7fabbb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:43.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:43.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:43.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 5 note: GC not initialized 2024-05-09T20:25:43.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fabba7e4640 20 reqs_thread_entry: start 2024-05-09T20:25:43.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:43.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 20 Filter name: none 2024-05-09T20:25:43.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fab2bfff640 20 reqs_thread_entry: start 2024-05-09T20:25:43.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-09T20:25:43.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:43.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-05-09T20:25:43.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-09T20:25:43.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 : miss 2024-05-09T20:25:43.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.149+0000 7fad07aad980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.153+0000 7fad07aad980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:43.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.153+0000 7fad07aad980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:43.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.153+0000 7fad07aad980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:43.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.153+0000 7fad07aad980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:43.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.153+0000 7fad07aad980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:43.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.153+0000 7fad07aad980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 2024-05-09T20:25:43.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.157+0000 7fabbb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:43.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.161+0000 7fad07aad980 20 remove_watcher() i=0 2024-05-09T20:25:43.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.161+0000 7fad07aad980 2 removed watcher, disabling cache 2024-05-09T20:25:43.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.161+0000 7fad07aad980 20 remove_watcher() i=1 2024-05-09T20:25:43.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.165+0000 7fad07aad980 20 remove_watcher() i=2 2024-05-09T20:25:43.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.165+0000 7fad07aad980 20 remove_watcher() i=3 2024-05-09T20:25:43.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.169+0000 7fad07aad980 20 remove_watcher() i=4 2024-05-09T20:25:43.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.169+0000 7fad07aad980 20 remove_watcher() i=5 2024-05-09T20:25:43.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.173+0000 7fad07aad980 20 remove_watcher() i=6 2024-05-09T20:25:43.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.173+0000 7fad07aad980 20 remove_watcher() i=7 2024-05-09T20:25:43.178 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin reshard list 2024-05-09T20:25:43.203 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:43.203 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:43.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.257+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.257+0000 7efe9978a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.257+0000 7efe9978a980 20 realm 2024-05-09T20:25:43.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.257+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.257+0000 7efe9978a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.257+0000 7efe9978a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:43.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.257+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.261+0000 7efe9978a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.261+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.261+0000 7efe9978a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:43.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.261+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.261+0000 7efe9978a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:43.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.261+0000 7efe9978a980 20 searching for the correct realm 2024-05-09T20:25:43.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.265+0000 7efe9729e640 0 -- 172.21.15.63:0/3219553072 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7efe90017030 msgr2=0x7efe9005baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:43.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.481+0000 7efe9729e640 0 -- 172.21.15.63:0/3219553072 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7efe90017030 msgr2=0x7efe9005baa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:43.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.493+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:43.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 zone default found 2024-05-09T20:25:43.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 4 Realm: () 2024-05-09T20:25:43.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:43.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:43.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:43.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 zonegroup default 2024-05-09T20:25:43.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 started sync module instance, tier type = 2024-05-09T20:25:43.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.497+0000 7efe9978a980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:43.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.517+0000 7efe9978a980 20 add_watcher() i=0 2024-05-09T20:25:43.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.517+0000 7efe9978a980 20 add_watcher() i=1 2024-05-09T20:25:43.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.517+0000 7efe9978a980 20 add_watcher() i=2 2024-05-09T20:25:43.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.517+0000 7efe9978a980 20 add_watcher() i=3 2024-05-09T20:25:43.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.517+0000 7efe9978a980 20 add_watcher() i=4 2024-05-09T20:25:43.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.517+0000 7efe9978a980 20 add_watcher() i=5 2024-05-09T20:25:43.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.517+0000 7efe9978a980 20 add_watcher() i=6 2024-05-09T20:25:43.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.521+0000 7efe9978a980 20 add_watcher() i=7 2024-05-09T20:25:43.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.521+0000 7efe9978a980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:43.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.525+0000 7efd4dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:43.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.525+0000 7efe9978a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:43.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.525+0000 7efe9978a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:43.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.525+0000 7efe9978a980 5 note: GC not initialized 2024-05-09T20:25:43.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.525+0000 7efd4cfe1640 20 reqs_thread_entry: start 2024-05-09T20:25:43.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.525+0000 7efe9978a980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:43.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.525+0000 7efe9978a980 20 Filter name: none 2024-05-09T20:25:43.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.525+0000 7efcc67fc640 20 reqs_thread_entry: start 2024-05-09T20:25:43.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.541+0000 7efd4dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:43.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.545+0000 7efe9978a980 20 remove_watcher() i=0 2024-05-09T20:25:43.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.545+0000 7efe9978a980 2 removed watcher, disabling cache 2024-05-09T20:25:43.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.545+0000 7efe9978a980 20 remove_watcher() i=1 2024-05-09T20:25:43.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.549+0000 7efe9978a980 20 remove_watcher() i=2 2024-05-09T20:25:43.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.549+0000 7efe9978a980 20 remove_watcher() i=3 2024-05-09T20:25:43.543 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.553+0000 7efe9978a980 20 remove_watcher() i=4 2024-05-09T20:25:43.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.553+0000 7efe9978a980 20 remove_watcher() i=5 2024-05-09T20:25:43.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.557+0000 7efe9978a980 20 remove_watcher() i=6 2024-05-09T20:25:43.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.557+0000 7efe9978a980 20 remove_watcher() i=7 2024-05-09T20:25:43.565 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard process non empty bucket 2024-05-09T20:25:43.565 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:43.565 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin reshard process 2024-05-09T20:25:43.595 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:43.595 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:43.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.649+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 realm 2024-05-09T20:25:43.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:43.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:43.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:43.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.653+0000 7fe93ebf7980 20 searching for the correct realm 2024-05-09T20:25:43.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.657+0000 7fe93cf0c640 0 -- 172.21.15.63:0/997414535 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fe924032060 msgr2=0x7fe92405b970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:43.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.869+0000 7fe93bf0a640 0 -- 172.21.15.63:0/997414535 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fe92c0a3c20 msgr2=0x7fe92c1090d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:43.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:43.876 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:43.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:43.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:43.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:43.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:43.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.885+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:43.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:43.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 zone default found 2024-05-09T20:25:43.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 4 Realm: () 2024-05-09T20:25:43.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:43.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:43.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:43.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 zonegroup default 2024-05-09T20:25:43.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:43.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 started sync module instance, tier type = 2024-05-09T20:25:43.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.889+0000 7fe93ebf7980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:43.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 20 add_watcher() i=0 2024-05-09T20:25:43.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 20 add_watcher() i=1 2024-05-09T20:25:43.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 20 add_watcher() i=2 2024-05-09T20:25:43.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 20 add_watcher() i=3 2024-05-09T20:25:43.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 20 add_watcher() i=4 2024-05-09T20:25:43.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 20 add_watcher() i=5 2024-05-09T20:25:43.906 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 20 add_watcher() i=6 2024-05-09T20:25:43.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 20 add_watcher() i=7 2024-05-09T20:25:43.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.917+0000 7fe93ebf7980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:43.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.921+0000 7fe7f2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:43.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.925+0000 7fe93ebf7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:43.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.925+0000 7fe93ebf7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:43.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.925+0000 7fe93ebf7980 5 note: GC not initialized 2024-05-09T20:25:43.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.925+0000 7fe7f1fdb640 20 reqs_thread_entry: start 2024-05-09T20:25:43.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.925+0000 7fe93ebf7980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:43.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.925+0000 7fe93ebf7980 20 Filter name: none 2024-05-09T20:25:43.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.925+0000 7fe7637fe640 20 reqs_thread_entry: start 2024-05-09T20:25:43.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.925+0000 7fe93ebf7980 20 processing logshard = reshard.0000000000 2024-05-09T20:25:43.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.929+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000000 , ret = 0 2024-05-09T20:25:43.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.929+0000 7fe93ebf7980 20 processing logshard = reshard.0000000001 2024-05-09T20:25:43.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.937+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000001 , ret = 0 2024-05-09T20:25:43.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.937+0000 7fe93ebf7980 20 processing logshard = reshard.0000000002 2024-05-09T20:25:43.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.941+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000002 , ret = 0 2024-05-09T20:25:43.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.941+0000 7fe93ebf7980 20 processing logshard = reshard.0000000003 2024-05-09T20:25:43.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.949+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000003 , ret = 0 2024-05-09T20:25:43.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.949+0000 7fe93ebf7980 20 processing logshard = reshard.0000000004 2024-05-09T20:25:43.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.953+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000004 , ret = 0 2024-05-09T20:25:43.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.953+0000 7fe93ebf7980 20 processing logshard = reshard.0000000005 2024-05-09T20:25:43.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.957+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000005 , ret = 0 2024-05-09T20:25:43.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 20 processing logshard = reshard.0000000006 2024-05-09T20:25:43.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 20 process_entry resharding a-bucket 2024-05-09T20:25:43.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-09T20:25:43.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:43.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x1 2024-05-09T20:25:43.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-09T20:25:43.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 : miss 2024-05-09T20:25:43.953 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.961+0000 7fe93ebf7980 20 rados->read ofs=0 len=0 2024-05-09T20:25:43.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.965+0000 7fe93ebf7980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:43.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.965+0000 7fe93ebf7980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:43.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.965+0000 7fe93ebf7980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:43.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.965+0000 7fe93ebf7980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:43.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.965+0000 7fe93ebf7980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:43.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.965+0000 7fe93ebf7980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 2024-05-09T20:25:43.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.977+0000 7fe93ebf7980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:43.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.977+0000 7fe93ebf7980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:43.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.977+0000 7fe93ebf7980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:43.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.977+0000 7fe93ebf7980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17, ofs0, ns] 2024-05-09T20:25:43.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.981+0000 7fe8137fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431578 cookie 94411303909968 notifier 5801 bl.length()=819 2024-05-09T20:25:43.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.981+0000 7fe8137fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:43.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.981+0000 7fe8137fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:43.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.981+0000 7fe8137fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:43.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.981+0000 7fe93ebf7980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_sources= new_sources= 2024-05-09T20:25:43.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.981+0000 7fe93ebf7980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential sources added=[] removed=[] 2024-05-09T20:25:43.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.981+0000 7fe93ebf7980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_dests= new_dests= 2024-05-09T20:25:43.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.981+0000 7fe93ebf7980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential dests added=[] removed=[] 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.0 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.0 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.1 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.1 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.2 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.2 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.3 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.3 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.4 2024-05-09T20:25:43.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.4 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.5 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.5 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.6 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.6 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.7 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.7 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.8 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.8 2024-05-09T20:25:43.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.9 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.9 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.10 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.10 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.11 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.11 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.12 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.12 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.0 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.0 2024-05-09T20:25:43.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.1 2024-05-09T20:25:43.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.989+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.1 2024-05-09T20:25:43.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.993+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.2 2024-05-09T20:25:43.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.993+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.2 2024-05-09T20:25:43.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.993+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.3 2024-05-09T20:25:43.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.993+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.3 2024-05-09T20:25:43.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.993+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.4 2024-05-09T20:25:43.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.993+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.4 2024-05-09T20:25:43.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.993+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.5 2024-05-09T20:25:43.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.993+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.5 2024-05-09T20:25:43.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.997+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.6 2024-05-09T20:25:43.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.997+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.6 2024-05-09T20:25:43.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.997+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.7 2024-05-09T20:25:43.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.997+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.7 2024-05-09T20:25:43.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.997+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.8 2024-05-09T20:25:43.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:43.997+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.8 2024-05-09T20:25:43.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.001+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.9 2024-05-09T20:25:43.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.001+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.9 2024-05-09T20:25:43.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.001+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.10 2024-05-09T20:25:43.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.001+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.10 2024-05-09T20:25:43.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.001+0000 7fe93ebf7980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.11 2024-05-09T20:25:43.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.001+0000 7fe93ebf7980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.1.11 2024-05-09T20:25:43.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.005+0000 7fe93ebf7980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 : hit (requested=0x17, cached=0x17) 2024-05-09T20:25:43.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.005+0000 7fe93ebf7980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 2024-05-09T20:25:43.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.009+0000 7fe93ebf7980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:43.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.009+0000 7fe93ebf7980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:43.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.009+0000 7fe93ebf7980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:43.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.009+0000 7fe93ebf7980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17, ofs0, ns] 2024-05-09T20:25:43.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.009+0000 7fe812ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431579 cookie 94411303909968 notifier 5801 bl.length()=787 2024-05-09T20:25:43.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.009+0000 7fe812ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:25:43.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.009+0000 7fe812ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:25:43.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.009+0000 7fe812ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:25:44.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.013+0000 7fe93ebf7980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_sources= new_sources= 2024-05-09T20:25:44.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.013+0000 7fe93ebf7980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential sources added=[] removed=[] 2024-05-09T20:25:44.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.013+0000 7fe93ebf7980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_dests= new_dests= 2024-05-09T20:25:44.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.013+0000 7fe93ebf7980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential dests added=[] removed=[] 2024-05-09T20:25:44.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.017+0000 7fe93ebf7980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-05-09T20:25:44.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.021+0000 7fe93ebf7980 20 process_entry removing reshard queue entry for bucket a-bucket 2024-05-09T20:25:44.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.025+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000006 , ret = 0 2024-05-09T20:25:44.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.025+0000 7fe93ebf7980 20 processing logshard = reshard.0000000007 2024-05-09T20:25:44.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.029+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000007 , ret = 0 2024-05-09T20:25:44.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.029+0000 7fe93ebf7980 20 processing logshard = reshard.0000000008 2024-05-09T20:25:44.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.037+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000008 , ret = 0 2024-05-09T20:25:44.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.037+0000 7fe93ebf7980 20 processing logshard = reshard.0000000009 2024-05-09T20:25:44.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.041+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000009 , ret = 0 2024-05-09T20:25:44.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.041+0000 7fe93ebf7980 20 processing logshard = reshard.0000000010 2024-05-09T20:25:44.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.045+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000010 , ret = 0 2024-05-09T20:25:44.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.045+0000 7fe93ebf7980 20 processing logshard = reshard.0000000011 2024-05-09T20:25:44.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.049+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000011 , ret = 0 2024-05-09T20:25:44.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.049+0000 7fe93ebf7980 20 processing logshard = reshard.0000000012 2024-05-09T20:25:44.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.057+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000012 , ret = 0 2024-05-09T20:25:44.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.057+0000 7fe93ebf7980 20 processing logshard = reshard.0000000013 2024-05-09T20:25:44.054 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.065+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000013 , ret = 0 2024-05-09T20:25:44.054 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.065+0000 7fe93ebf7980 20 processing logshard = reshard.0000000014 2024-05-09T20:25:44.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.069+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000014 , ret = 0 2024-05-09T20:25:44.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.069+0000 7fe93ebf7980 20 processing logshard = reshard.0000000015 2024-05-09T20:25:44.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.073+0000 7fe93ebf7980 20 finish processing logshard = reshard.0000000015 , ret = 0 2024-05-09T20:25:44.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.077+0000 7fe7f2fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:44.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.085+0000 7fe93ebf7980 20 remove_watcher() i=0 2024-05-09T20:25:44.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.085+0000 7fe93ebf7980 2 removed watcher, disabling cache 2024-05-09T20:25:44.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.085+0000 7fe93ebf7980 20 remove_watcher() i=1 2024-05-09T20:25:44.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.089+0000 7fe93ebf7980 20 remove_watcher() i=2 2024-05-09T20:25:44.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.093+0000 7fe93ebf7980 20 remove_watcher() i=3 2024-05-09T20:25:44.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.093+0000 7fe93ebf7980 20 remove_watcher() i=4 2024-05-09T20:25:44.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.097+0000 7fe93ebf7980 20 remove_watcher() i=5 2024-05-09T20:25:44.087 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.097+0000 7fe93ebf7980 20 remove_watcher() i=6 2024-05-09T20:25:44.089 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.097+0000 7fe93ebf7980 20 remove_watcher() i=7 2024-05-09T20:25:44.107 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-09T20:25:44.138 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:44.139 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:44.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.193+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:44.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 realm 2024-05-09T20:25:44.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:44.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:44.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:44.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:44.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:44.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.197+0000 7f299d9fc980 20 searching for the correct realm 2024-05-09T20:25:44.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.201+0000 7f299b510640 0 -- 172.21.15.63:0/3185361162 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f29940b5c10 msgr2=0x7f29940cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:44.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.417+0000 7f299b510640 0 -- 172.21.15.63:0/3185361162 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f29940b5c10 msgr2=0x7f29940cdab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:44.419 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:44.419 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:44.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:44.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:44.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:44.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:44.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:44.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:44.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.429+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:44.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 zone default found 2024-05-09T20:25:44.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 4 Realm: () 2024-05-09T20:25:44.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:44.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:44.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:44.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 zonegroup default 2024-05-09T20:25:44.422 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:44.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:44.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:44.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 started sync module instance, tier type = 2024-05-09T20:25:44.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.433+0000 7f299d9fc980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:44.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 20 add_watcher() i=0 2024-05-09T20:25:44.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 20 add_watcher() i=1 2024-05-09T20:25:44.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 20 add_watcher() i=2 2024-05-09T20:25:44.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 20 add_watcher() i=3 2024-05-09T20:25:44.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 20 add_watcher() i=4 2024-05-09T20:25:44.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 20 add_watcher() i=5 2024-05-09T20:25:44.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 20 add_watcher() i=6 2024-05-09T20:25:44.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 20 add_watcher() i=7 2024-05-09T20:25:44.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.457+0000 7f299d9fc980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:44.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.461+0000 7f2851fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:44.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:44.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:44.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 5 note: GC not initialized 2024-05-09T20:25:44.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f2850fe1640 20 reqs_thread_entry: start 2024-05-09T20:25:44.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:44.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 Filter name: none 2024-05-09T20:25:44.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f27ca7fc640 20 reqs_thread_entry: start 2024-05-09T20:25:44.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:44.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:44.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:44.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.465+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.469+0000 7f299d9fc980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:25:44.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.469+0000 7f299d9fc980 20 rados->read ofs=0 len=0 2024-05-09T20:25:44.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.469+0000 7f299d9fc980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:25:44.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.469+0000 7f299d9fc980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:44.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.473+0000 7f2851fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:44.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.477+0000 7f299d9fc980 20 remove_watcher() i=0 2024-05-09T20:25:44.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.477+0000 7f299d9fc980 2 removed watcher, disabling cache 2024-05-09T20:25:44.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.477+0000 7f299d9fc980 20 remove_watcher() i=1 2024-05-09T20:25:44.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.481+0000 7f299d9fc980 20 remove_watcher() i=2 2024-05-09T20:25:44.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.485+0000 7f299d9fc980 20 remove_watcher() i=3 2024-05-09T20:25:44.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.485+0000 7f299d9fc980 20 remove_watcher() i=4 2024-05-09T20:25:44.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.489+0000 7f299d9fc980 20 remove_watcher() i=5 2024-05-09T20:25:44.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.489+0000 7f299d9fc980 20 remove_watcher() i=6 2024-05-09T20:25:44.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:44.489+0000 7f299d9fc980 20 remove_watcher() i=7 2024-05-09T20:25:44.495 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:44.496 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:44.496 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: a-bucket id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 num_objects: 8 size_kb: 1 num_shards: 13 2024-05-09T20:25:44.496 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:44.496 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with EIO injected at set_target_layout 2024-05-09T20:25:44.496 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:46.544 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-09T20:25:46.579 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:46.579 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:46.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.633+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:46.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 realm 2024-05-09T20:25:46.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:46.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:46.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:46.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:46.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.637+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.641+0000 7fa5a3aac980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:46.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.641+0000 7fa5a3aac980 20 searching for the correct realm 2024-05-09T20:25:46.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.645+0000 7fa5a15c0640 0 -- 172.21.15.63:0/2661187082 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fa59c0c5560 msgr2=0x7fa59c0c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:25:46.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.861+0000 7fa5a15c0640 0 -- 172.21.15.63:0/2661187082 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fa59c0c5560 msgr2=0x7fa59c0c59a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:46.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:46.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:46.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 zone default found 2024-05-09T20:25:46.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 4 Realm: () 2024-05-09T20:25:46.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:46.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:46.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:46.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 zonegroup default 2024-05-09T20:25:46.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.873+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.877+0000 7fa5a3aac980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:46.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.877+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.877+0000 7fa5a3aac980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:46.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.877+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.877+0000 7fa5a3aac980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:46.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.877+0000 7fa5a3aac980 20 started sync module instance, tier type = 2024-05-09T20:25:46.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.877+0000 7fa5a3aac980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:46.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.901+0000 7fa5a3aac980 20 add_watcher() i=0 2024-05-09T20:25:46.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.901+0000 7fa5a3aac980 20 add_watcher() i=1 2024-05-09T20:25:46.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.901+0000 7fa5a3aac980 20 add_watcher() i=2 2024-05-09T20:25:46.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.901+0000 7fa5a3aac980 20 add_watcher() i=3 2024-05-09T20:25:46.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.901+0000 7fa5a3aac980 20 add_watcher() i=4 2024-05-09T20:25:46.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.901+0000 7fa5a3aac980 20 add_watcher() i=5 2024-05-09T20:25:46.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.901+0000 7fa5a3aac980 20 add_watcher() i=6 2024-05-09T20:25:46.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.905+0000 7fa5a3aac980 20 add_watcher() i=7 2024-05-09T20:25:46.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.905+0000 7fa5a3aac980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:46.898 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa4577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:46.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa5a3aac980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:46.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa5a3aac980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:46.899 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa5a3aac980 5 note: GC not initialized 2024-05-09T20:25:46.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa4567e4640 20 reqs_thread_entry: start 2024-05-09T20:25:46.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa5a3aac980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:46.900 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa5a3aac980 20 Filter name: none 2024-05-09T20:25:46.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa3c7fff640 20 reqs_thread_entry: start 2024-05-09T20:25:46.901 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.909+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:46.902 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:46.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:46.903 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:46.904 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 rados->read ofs=0 len=0 2024-05-09T20:25:46.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:46.905 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.913+0000 7fa5a3aac980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:46.908 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.917+0000 7fa4577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:46.912 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.921+0000 7fa5a3aac980 20 remove_watcher() i=0 2024-05-09T20:25:46.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.921+0000 7fa5a3aac980 2 removed watcher, disabling cache 2024-05-09T20:25:46.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.925+0000 7fa5a3aac980 20 remove_watcher() i=1 2024-05-09T20:25:46.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.929+0000 7fa5a3aac980 20 remove_watcher() i=2 2024-05-09T20:25:46.920 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.929+0000 7fa5a3aac980 20 remove_watcher() i=3 2024-05-09T20:25:46.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.933+0000 7fa5a3aac980 20 remove_watcher() i=4 2024-05-09T20:25:46.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.933+0000 7fa5a3aac980 20 remove_watcher() i=5 2024-05-09T20:25:46.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.937+0000 7fa5a3aac980 20 remove_watcher() i=6 2024-05-09T20:25:46.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:46.937+0000 7fa5a3aac980 20 remove_watcher() i=7 2024-05-09T20:25:46.944 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:46.944 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:46.944 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-set-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:25:46.944 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:46.944 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout 2024-05-09T20:25:46.983 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:46.983 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:47.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 20 realm 2024-05-09T20:25:47.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:47.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.037+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.041+0000 7f48fc55c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:47.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.041+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.041+0000 7f48fc55c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:47.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.041+0000 7f48fc55c980 20 searching for the correct realm 2024-05-09T20:25:47.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.045+0000 7f48fa070640 0 -- 172.21.15.63:0/3441051148 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f48f403e060 msgr2=0x7f48f4063500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:47.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.261+0000 7f48fa070640 0 -- 172.21.15.63:0/3441051148 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f48f403e060 msgr2=0x7f48f4063500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:47.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:47.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:47.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:47.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:47.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:47.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:47.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:47.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.273+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:47.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 zone default found 2024-05-09T20:25:47.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 4 Realm: () 2024-05-09T20:25:47.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:47.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:47.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:47.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 zonegroup default 2024-05-09T20:25:47.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 started sync module instance, tier type = 2024-05-09T20:25:47.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.277+0000 7f48fc55c980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:47.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 20 add_watcher() i=0 2024-05-09T20:25:47.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 20 add_watcher() i=1 2024-05-09T20:25:47.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 20 add_watcher() i=2 2024-05-09T20:25:47.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 20 add_watcher() i=3 2024-05-09T20:25:47.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 20 add_watcher() i=4 2024-05-09T20:25:47.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 20 add_watcher() i=5 2024-05-09T20:25:47.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 20 add_watcher() i=6 2024-05-09T20:25:47.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 20 add_watcher() i=7 2024-05-09T20:25:47.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.305+0000 7f48fc55c980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:47.300 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.309+0000 7f47b17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:47.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:47.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:47.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 5 note: GC not initialized 2024-05-09T20:25:47.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f47ab7fe640 20 reqs_thread_entry: start 2024-05-09T20:25:47.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:47.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 20 Filter name: none 2024-05-09T20:25:47.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f47a97fa640 20 reqs_thread_entry: start 2024-05-09T20:25:47.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-05-09T20:25:47.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:47.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-05-09T20:25:47.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-05-09T20:25:47.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 : miss 2024-05-09T20:25:47.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:47.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 info.flags=0x17 2024-05-09T20:25:47.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 to cache LRU end 2024-05-09T20:25:47.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:47.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:47.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.313+0000 7f48fc55c980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 2024-05-09T20:25:47.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.321+0000 7f48fc55c980 -1 Injecting error=-5 at location=set_target_layout 2024-05-09T20:25:47.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.321+0000 7f48fc55c980 0 ERROR: init_target_layout failed to write target index layout to bucket info: (5) Input/output error 2024-05-09T20:25:47.321 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.333+0000 7f47b17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:47.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.337+0000 7f48fc55c980 20 remove_watcher() i=0 2024-05-09T20:25:47.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.337+0000 7f48fc55c980 2 removed watcher, disabling cache 2024-05-09T20:25:47.328 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.337+0000 7f48fc55c980 20 remove_watcher() i=1 2024-05-09T20:25:47.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.341+0000 7f48fc55c980 20 remove_watcher() i=2 2024-05-09T20:25:47.334 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.345+0000 7f48fc55c980 20 remove_watcher() i=3 2024-05-09T20:25:47.336 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.345+0000 7f48fc55c980 20 remove_watcher() i=4 2024-05-09T20:25:47.337 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.349+0000 7f48fc55c980 20 remove_watcher() i=5 2024-05-09T20:25:47.339 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.349+0000 7f48fc55c980 20 remove_watcher() i=6 2024-05-09T20:25:47.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.349+0000 7f48fc55c980 20 remove_watcher() i=7 2024-05-09T20:25:47.357 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-09T20:25:47.390 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:47.390 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:47.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 20 realm 2024-05-09T20:25:47.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:47.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.445+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.449+0000 7efc9467a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:47.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.449+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.449+0000 7efc9467a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:47.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.449+0000 7efc9467a980 20 searching for the correct realm 2024-05-09T20:25:47.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.453+0000 7efc9298f640 0 -- 172.21.15.63:0/3506138790 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7efc88008040 msgr2=0x7efc880919d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:47.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.461+0000 7efc9198d640 0 -- 172.21.15.63:0/3506138790 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7efc8405ecc0 msgr2=0x7efc84054a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:47.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:47.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:47.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:47.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:47.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:47.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:47.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 zone default found 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 4 Realm: () 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 zonegroup default 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.469+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.473+0000 7efc9467a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.473+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.473+0000 7efc9467a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.473+0000 7efc9467a980 20 started sync module instance, tier type = 2024-05-09T20:25:47.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.473+0000 7efc9467a980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:47.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 20 add_watcher() i=0 2024-05-09T20:25:47.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 20 add_watcher() i=1 2024-05-09T20:25:47.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 20 add_watcher() i=2 2024-05-09T20:25:47.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 20 add_watcher() i=3 2024-05-09T20:25:47.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 20 add_watcher() i=4 2024-05-09T20:25:47.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 20 add_watcher() i=5 2024-05-09T20:25:47.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 20 add_watcher() i=6 2024-05-09T20:25:47.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 20 add_watcher() i=7 2024-05-09T20:25:47.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efc9467a980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:47.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.489+0000 7efb48fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:47.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:47.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:47.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 5 note: GC not initialized 2024-05-09T20:25:47.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efac37fe640 20 reqs_thread_entry: start 2024-05-09T20:25:47.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:47.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 Filter name: none 2024-05-09T20:25:47.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efac17fa640 20 reqs_thread_entry: start 2024-05-09T20:25:47.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:47.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:47.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:47.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:47.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:47.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.493+0000 7efc9467a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:47.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.497+0000 7efb48fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:47.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.501+0000 7efc9467a980 20 remove_watcher() i=0 2024-05-09T20:25:47.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.501+0000 7efc9467a980 2 removed watcher, disabling cache 2024-05-09T20:25:47.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.505+0000 7efc9467a980 20 remove_watcher() i=1 2024-05-09T20:25:47.497 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.505+0000 7efc9467a980 20 remove_watcher() i=2 2024-05-09T20:25:47.499 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.509+0000 7efc9467a980 20 remove_watcher() i=3 2024-05-09T20:25:47.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.509+0000 7efc9467a980 20 remove_watcher() i=4 2024-05-09T20:25:47.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.513+0000 7efc9467a980 20 remove_watcher() i=5 2024-05-09T20:25:47.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.513+0000 7efc9467a980 20 remove_watcher() i=6 2024-05-09T20:25:47.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.513+0000 7efc9467a980 20 remove_watcher() i=7 2024-05-09T20:25:47.519 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:47.519 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:47.519 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-set-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:25:47.519 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:47.619 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:25:47.656 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:47.656 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:47.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.709+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.709+0000 7f7d5fd9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.709+0000 7f7d5fd9f980 20 realm 2024-05-09T20:25:47.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.709+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.709+0000 7f7d5fd9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.709+0000 7f7d5fd9f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:47.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.709+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.713+0000 7f7d5fd9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:47.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.713+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.713+0000 7f7d5fd9f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:47.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.713+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:47.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.713+0000 7f7d5fd9f980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:47.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.713+0000 7f7d5fd9f980 20 searching for the correct realm 2024-05-09T20:25:47.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.717+0000 7f7d5d0b2640 0 -- 172.21.15.63:0/1117559511 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f7d500d4be0 msgr2=0x7f7d500b9450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:47.919 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:47.929+0000 7f7d5d0b2640 0 -- 172.21.15.63:0/1117559511 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f7d500c4910 msgr2=0x7f7d500f1400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:48.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:48.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 zone default found 2024-05-09T20:25:48.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 4 Realm: () 2024-05-09T20:25:48.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:48.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:48.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:48.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 zonegroup default 2024-05-09T20:25:48.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.153+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.157+0000 7f7d5fd9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.157+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.157+0000 7f7d5fd9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.157+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.157+0000 7f7d5fd9f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.157+0000 7f7d5fd9f980 20 started sync module instance, tier type = 2024-05-09T20:25:48.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.157+0000 7f7d5fd9f980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.181+0000 7f7d5fd9f980 20 add_watcher() i=0 2024-05-09T20:25:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.181+0000 7f7d5fd9f980 20 add_watcher() i=1 2024-05-09T20:25:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.181+0000 7f7d5fd9f980 20 add_watcher() i=2 2024-05-09T20:25:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.181+0000 7f7d5fd9f980 20 add_watcher() i=3 2024-05-09T20:25:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.181+0000 7f7d5fd9f980 20 add_watcher() i=4 2024-05-09T20:25:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.181+0000 7f7d5fd9f980 20 add_watcher() i=5 2024-05-09T20:25:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.181+0000 7f7d5fd9f980 20 add_watcher() i=6 2024-05-09T20:25:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.185+0000 7f7d5fd9f980 20 add_watcher() i=7 2024-05-09T20:25:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.185+0000 7f7d5fd9f980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:48.179 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.189+0000 7f7c13fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:48.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.189+0000 7f7d5fd9f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:48.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.189+0000 7f7d5fd9f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:48.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.189+0000 7f7d5fd9f980 5 note: GC not initialized 2024-05-09T20:25:48.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.189+0000 7f7c12fe5640 20 reqs_thread_entry: start 2024-05-09T20:25:48.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.189+0000 7f7d5fd9f980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:48.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.189+0000 7f7d5fd9f980 20 Filter name: none 2024-05-09T20:25:48.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.189+0000 7f7c10fe1640 20 reqs_thread_entry: start 2024-05-09T20:25:48.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-05-09T20:25:48.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:48.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-05-09T20:25:48.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-05-09T20:25:48.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 : miss 2024-05-09T20:25:48.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:48.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 info.flags=0x17 2024-05-09T20:25:48.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 to cache LRU end 2024-05-09T20:25:48.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:48.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:48.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.193+0000 7f7d5fd9f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 2024-05-09T20:25:48.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.201+0000 7f7d5fd9f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 info.flags=0x17 2024-05-09T20:25:48.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.201+0000 7f7d5fd9f980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 to cache LRU end 2024-05-09T20:25:48.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.201+0000 7f7d5fd9f980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:48.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.201+0000 7f7d5fd9f980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19, ofs0, ns] 2024-05-09T20:25:48.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.205+0000 7f7c417fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431581 cookie 93919283844720 notifier 5885 bl.length()=917 2024-05-09T20:25:48.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.205+0000 7f7c417fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 info.flags=0x17 2024-05-09T20:25:48.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.205+0000 7f7c417fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 to cache LRU end 2024-05-09T20:25:48.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.205+0000 7f7c417fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:48.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.205+0000 7f7d5fd9f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19]): orig_sources= new_sources= 2024-05-09T20:25:48.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.205+0000 7f7d5fd9f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19]): potential sources added=[] removed=[] 2024-05-09T20:25:48.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.205+0000 7f7d5fd9f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19]): orig_dests= new_dests= 2024-05-09T20:25:48.196 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.205+0000 7f7d5fd9f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19]): potential dests added=[] removed=[] 2024-05-09T20:25:48.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.0 2024-05-09T20:25:48.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.0 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.1 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.1 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.2 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.2 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.3 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.3 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.4 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.4 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.5 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.5 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.6 2024-05-09T20:25:48.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.6 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.7 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.7 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.8 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.8 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.9 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.9 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.10 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.10 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.11 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1.11 2024-05-09T20:25:48.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.0 2024-05-09T20:25:48.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.1 2024-05-09T20:25:48.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.2 2024-05-09T20:25:48.204 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.213+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.3 2024-05-09T20:25:48.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.217+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.4 2024-05-09T20:25:48.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.217+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.5 2024-05-09T20:25:48.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.217+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.6 2024-05-09T20:25:48.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.221+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.7 2024-05-09T20:25:48.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.221+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.8 2024-05-09T20:25:48.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.221+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.9 2024-05-09T20:25:48.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.221+0000 7f7d5fd9f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19.10 2024-05-09T20:25:48.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.229+0000 7f7d5fd9f980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 : hit (requested=0x17, cached=0x17) 2024-05-09T20:25:48.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.229+0000 7f7d5fd9f980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 2024-05-09T20:25:48.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.229+0000 7f7d5fd9f980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 info.flags=0x17 2024-05-09T20:25:48.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.229+0000 7f7d5fd9f980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 to cache LRU end 2024-05-09T20:25:48.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.229+0000 7f7d5fd9f980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:48.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.229+0000 7f7d5fd9f980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19, ofs0, ns] 2024-05-09T20:25:48.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.233+0000 7f7c40ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431582 cookie 93919283844720 notifier 5885 bl.length()=885 2024-05-09T20:25:48.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.233+0000 7f7c40ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 info.flags=0x17 2024-05-09T20:25:48.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.233+0000 7f7c40ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 to cache LRU end 2024-05-09T20:25:48.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.233+0000 7f7c40ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:48.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.233+0000 7f7d5fd9f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19]): orig_sources= new_sources= 2024-05-09T20:25:48.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.233+0000 7f7d5fd9f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19]): potential sources added=[] removed=[] 2024-05-09T20:25:48.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.233+0000 7f7d5fd9f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19]): orig_dests= new_dests= 2024-05-09T20:25:48.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.233+0000 7f7d5fd9f980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19]): potential dests added=[] removed=[] 2024-05-09T20:25:48.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.237+0000 7f7d5fd9f980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-05-09T20:25:48.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.241+0000 7f7c13fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:48.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.249+0000 7f7d5fd9f980 20 remove_watcher() i=0 2024-05-09T20:25:48.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.249+0000 7f7d5fd9f980 2 removed watcher, disabling cache 2024-05-09T20:25:48.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.249+0000 7f7d5fd9f980 20 remove_watcher() i=1 2024-05-09T20:25:48.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.253+0000 7f7d5fd9f980 20 remove_watcher() i=2 2024-05-09T20:25:48.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.253+0000 7f7d5fd9f980 20 remove_watcher() i=3 2024-05-09T20:25:48.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.257+0000 7f7d5fd9f980 20 remove_watcher() i=4 2024-05-09T20:25:48.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.261+0000 7f7d5fd9f980 20 remove_watcher() i=5 2024-05-09T20:25:48.251 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.261+0000 7f7d5fd9f980 20 remove_watcher() i=6 2024-05-09T20:25:48.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.261+0000 7f7d5fd9f980 20 remove_watcher() i=7 2024-05-09T20:25:48.271 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-09T20:25:48.303 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:48.303 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:48.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 20 realm 2024-05-09T20:25:48.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:48.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.361+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.365+0000 7f901d80c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:48.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.365+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.365+0000 7f901d80c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:48.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.365+0000 7f901d80c980 20 searching for the correct realm 2024-05-09T20:25:48.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.369+0000 7f901b320640 0 -- 172.21.15.63:0/540323301 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f901402d040 msgr2=0x7f901402d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:48.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.585+0000 7f901b320640 0 -- 172.21.15.63:0/540323301 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f901402d040 msgr2=0x7f901402d930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:48.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:48.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:48.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:48.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:48.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:48.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:48.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:48.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.597+0000 7f901d80c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:48.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 zone default found 2024-05-09T20:25:48.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 4 Realm: () 2024-05-09T20:25:48.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:48.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:48.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:48.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 zonegroup default 2024-05-09T20:25:48.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:48.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 started sync module instance, tier type = 2024-05-09T20:25:48.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.601+0000 7f901d80c980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:48.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.625+0000 7f901d80c980 20 add_watcher() i=0 2024-05-09T20:25:48.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.625+0000 7f901d80c980 20 add_watcher() i=1 2024-05-09T20:25:48.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.625+0000 7f901d80c980 20 add_watcher() i=2 2024-05-09T20:25:48.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.625+0000 7f901d80c980 20 add_watcher() i=3 2024-05-09T20:25:48.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.625+0000 7f901d80c980 20 add_watcher() i=4 2024-05-09T20:25:48.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.625+0000 7f901d80c980 20 add_watcher() i=5 2024-05-09T20:25:48.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.625+0000 7f901d80c980 20 add_watcher() i=6 2024-05-09T20:25:48.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.629+0000 7f901d80c980 20 add_watcher() i=7 2024-05-09T20:25:48.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.629+0000 7f901d80c980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:48.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.633+0000 7f8ed1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:48.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.633+0000 7f901d80c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:48.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.633+0000 7f901d80c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:48.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.633+0000 7f901d80c980 5 note: GC not initialized 2024-05-09T20:25:48.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.633+0000 7f8ed0fe1640 20 reqs_thread_entry: start 2024-05-09T20:25:48.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.633+0000 7f901d80c980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:48.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.633+0000 7f901d80c980 20 Filter name: none 2024-05-09T20:25:48.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f8e4a7fc640 20 reqs_thread_entry: start 2024-05-09T20:25:48.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f901d80c980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:48.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f901d80c980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:48.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f901d80c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:48.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f901d80c980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:48.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.637+0000 7f901d80c980 20 rados->read ofs=0 len=0 2024-05-09T20:25:48.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.641+0000 7f901d80c980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:48.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.641+0000 7f901d80c980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:48.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.641+0000 7f8ed1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:48.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.649+0000 7f901d80c980 20 remove_watcher() i=0 2024-05-09T20:25:48.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.649+0000 7f901d80c980 2 removed watcher, disabling cache 2024-05-09T20:25:48.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.649+0000 7f901d80c980 20 remove_watcher() i=1 2024-05-09T20:25:48.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.653+0000 7f901d80c980 20 remove_watcher() i=2 2024-05-09T20:25:48.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.653+0000 7f901d80c980 20 remove_watcher() i=3 2024-05-09T20:25:48.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.657+0000 7f901d80c980 20 remove_watcher() i=4 2024-05-09T20:25:48.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.657+0000 7f901d80c980 20 remove_watcher() i=5 2024-05-09T20:25:48.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.661+0000 7f901d80c980 20 remove_watcher() i=6 2024-05-09T20:25:48.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:48.661+0000 7f901d80c980 20 remove_watcher() i=7 2024-05-09T20:25:48.667 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:48.667 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:48.667 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-set-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.19 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-09T20:25:48.667 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:49.574 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with ECANCELED injected at set_target_layout 2024-05-09T20:25:49.574 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:51.688 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-09T20:25:51.722 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:51.722 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:51.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.777+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:51.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:51.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 realm 2024-05-09T20:25:51.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:51.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:51.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:51.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:51.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:51.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:51.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:51.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:51.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:51.773 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.781+0000 7fbaaaf0f980 20 searching for the correct realm 2024-05-09T20:25:51.774 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.785+0000 7fbaa8222640 0 -- 172.21.15.63:0/778054633 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fba900022b0 msgr2=0x7fba9801a1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:25:51.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:51.997+0000 7fbaa8222640 0 -- 172.21.15.63:0/778054633 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7fba9805d810 msgr2=0x7fba98054bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:52.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:52.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:52.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:52.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:52.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:52.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:52.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:52.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:52.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 zone default found 2024-05-09T20:25:52.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 4 Realm: () 2024-05-09T20:25:52.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:52.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:52.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:52.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 zonegroup default 2024-05-09T20:25:52.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.221+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.225+0000 7fbaaaf0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.225+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.225+0000 7fbaaaf0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.225+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.225+0000 7fbaaaf0f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.225+0000 7fbaaaf0f980 20 started sync module instance, tier type = 2024-05-09T20:25:52.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.225+0000 7fbaaaf0f980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:52.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.249+0000 7fbaaaf0f980 20 add_watcher() i=0 2024-05-09T20:25:52.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.249+0000 7fbaaaf0f980 20 add_watcher() i=1 2024-05-09T20:25:52.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.249+0000 7fbaaaf0f980 20 add_watcher() i=2 2024-05-09T20:25:52.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.249+0000 7fbaaaf0f980 20 add_watcher() i=3 2024-05-09T20:25:52.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.249+0000 7fbaaaf0f980 20 add_watcher() i=4 2024-05-09T20:25:52.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.249+0000 7fbaaaf0f980 20 add_watcher() i=5 2024-05-09T20:25:52.241 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.249+0000 7fbaaaf0f980 20 add_watcher() i=6 2024-05-09T20:25:52.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.253+0000 7fbaaaf0f980 20 add_watcher() i=7 2024-05-09T20:25:52.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.253+0000 7fbaaaf0f980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:52.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fb95efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:52.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fbaaaf0f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:52.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fbaaaf0f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:52.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fbaaaf0f980 5 note: GC not initialized 2024-05-09T20:25:52.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fb95dfe3640 20 reqs_thread_entry: start 2024-05-09T20:25:52.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fbaaaf0f980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:52.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fbaaaf0f980 20 Filter name: none 2024-05-09T20:25:52.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fb8cf7fe640 20 reqs_thread_entry: start 2024-05-09T20:25:52.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.257+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:52.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:52.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:52.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:52.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:52.253 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.261+0000 7fbaaaf0f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:52.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.265+0000 7fb95efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:52.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.273+0000 7fbaaaf0f980 20 remove_watcher() i=0 2024-05-09T20:25:52.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.273+0000 7fbaaaf0f980 2 removed watcher, disabling cache 2024-05-09T20:25:52.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.273+0000 7fbaaaf0f980 20 remove_watcher() i=1 2024-05-09T20:25:52.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.277+0000 7fbaaaf0f980 20 remove_watcher() i=2 2024-05-09T20:25:52.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.281+0000 7fbaaaf0f980 20 remove_watcher() i=3 2024-05-09T20:25:52.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.285+0000 7fbaaaf0f980 20 remove_watcher() i=4 2024-05-09T20:25:52.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.285+0000 7fbaaaf0f980 20 remove_watcher() i=5 2024-05-09T20:25:52.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.289+0000 7fbaaaf0f980 20 remove_watcher() i=6 2024-05-09T20:25:52.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.289+0000 7fbaaaf0f980 20 remove_watcher() i=7 2024-05-09T20:25:52.293 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:52.294 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:52.294 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-set-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:25:52.294 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:52.294 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at set_target_layout --inject-error-code 125 2024-05-09T20:25:52.329 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:52.329 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:52.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.381+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 realm 2024-05-09T20:25:52.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:52.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:52.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:52.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.385+0000 7f91494cb980 20 searching for the correct realm 2024-05-09T20:25:52.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.389+0000 7f91477e0640 0 -- 172.21.15.63:0/2258112721 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f913c02a070 msgr2=0x7f913c05e950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:25:52.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.601+0000 7f91467de640 0 -- 172.21.15.63:0/2258112721 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f914005b780 msgr2=0x7f91400268e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:52.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.617+0000 7f91494cb980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:52.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.617+0000 7f91494cb980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:52.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.617+0000 7f91494cb980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:52.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.617+0000 7f91494cb980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:52.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.617+0000 7f91494cb980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:52.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.617+0000 7f91494cb980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:52.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.617+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:52.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:52.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 zone default found 2024-05-09T20:25:52.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 4 Realm: () 2024-05-09T20:25:52.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:52.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:52.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:52.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 zonegroup default 2024-05-09T20:25:52.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.621+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.625+0000 7f91494cb980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.625+0000 7f91494cb980 20 started sync module instance, tier type = 2024-05-09T20:25:52.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.625+0000 7f91494cb980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:52.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 20 add_watcher() i=0 2024-05-09T20:25:52.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 20 add_watcher() i=1 2024-05-09T20:25:52.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 20 add_watcher() i=2 2024-05-09T20:25:52.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 20 add_watcher() i=3 2024-05-09T20:25:52.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 20 add_watcher() i=4 2024-05-09T20:25:52.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 20 add_watcher() i=5 2024-05-09T20:25:52.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 20 add_watcher() i=6 2024-05-09T20:25:52.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 20 add_watcher() i=7 2024-05-09T20:25:52.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.649+0000 7f91494cb980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:52.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f8ffd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:52.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:52.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:52.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 5 note: GC not initialized 2024-05-09T20:25:52.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f8f77fff640 20 reqs_thread_entry: start 2024-05-09T20:25:52.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:52.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 20 Filter name: none 2024-05-09T20:25:52.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f8f75ffb640 20 reqs_thread_entry: start 2024-05-09T20:25:52.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 10 cache get: name=default.rgw.meta+root+error-at-set-target-layout : miss 2024-05-09T20:25:52.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:52.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 10 cache put: name=default.rgw.meta+root+error-at-set-target-layout info.flags=0x11 2024-05-09T20:25:52.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 10 adding default.rgw.meta+root+error-at-set-target-layout to cache LRU end 2024-05-09T20:25:52.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 : miss 2024-05-09T20:25:52.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.657+0000 7f91494cb980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.661+0000 7f91494cb980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:52.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.661+0000 7f91494cb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 info.flags=0x17 2024-05-09T20:25:52.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.661+0000 7f91494cb980 10 adding default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 to cache LRU end 2024-05-09T20:25:52.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.661+0000 7f91494cb980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:52.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.661+0000 7f91494cb980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:52.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.661+0000 7f91494cb980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 2024-05-09T20:25:52.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.669+0000 7f91494cb980 -1 Injecting error=-125 at location=set_target_layout 2024-05-09T20:25:52.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.669+0000 7f91494cb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 info.flags=0x17 2024-05-09T20:25:52.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.669+0000 7f91494cb980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 to cache LRU end 2024-05-09T20:25:52.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.669+0000 7f91494cb980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:52.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.669+0000 7f91494cb980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20, ofs0, ns] 2024-05-09T20:25:52.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.673+0000 7f9029ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431585 cookie 94802894911328 notifier 6276 bl.length()=917 2024-05-09T20:25:52.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.673+0000 7f9029ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 info.flags=0x17 2024-05-09T20:25:52.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.673+0000 7f9029ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 to cache LRU end 2024-05-09T20:25:52.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.673+0000 7f9029ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:52.664 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.673+0000 7f91494cb980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20]): orig_sources= new_sources= 2024-05-09T20:25:52.664 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.673+0000 7f91494cb980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20]): potential sources added=[] removed=[] 2024-05-09T20:25:52.664 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.673+0000 7f91494cb980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20]): orig_dests= new_dests= 2024-05-09T20:25:52.664 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.673+0000 7f91494cb980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20]): potential dests added=[] removed=[] 2024-05-09T20:25:52.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.0 2024-05-09T20:25:52.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.0 2024-05-09T20:25:52.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.1 2024-05-09T20:25:52.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.1 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.2 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.2 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.3 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.3 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.4 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.4 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.5 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.5 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.6 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.6 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.7 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.7 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.8 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.8 2024-05-09T20:25:52.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.9 2024-05-09T20:25:52.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.9 2024-05-09T20:25:52.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.10 2024-05-09T20:25:52.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.10 2024-05-09T20:25:52.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.11 2024-05-09T20:25:52.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1.11 2024-05-09T20:25:52.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.0 2024-05-09T20:25:52.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.1 2024-05-09T20:25:52.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.681+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.2 2024-05-09T20:25:52.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.685+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.3 2024-05-09T20:25:52.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.685+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.4 2024-05-09T20:25:52.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.685+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.5 2024-05-09T20:25:52.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.685+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.6 2024-05-09T20:25:52.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.689+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.7 2024-05-09T20:25:52.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.689+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.8 2024-05-09T20:25:52.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.689+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.9 2024-05-09T20:25:52.681 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.689+0000 7f91494cb980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20.10 2024-05-09T20:25:52.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f91494cb980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 : hit (requested=0x17, cached=0x17) 2024-05-09T20:25:52.689 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f91494cb980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 2024-05-09T20:25:52.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f91494cb980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 info.flags=0x17 2024-05-09T20:25:52.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f91494cb980 10 moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 to cache LRU end 2024-05-09T20:25:52.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f91494cb980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:52.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f91494cb980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20, ofs0, ns] 2024-05-09T20:25:52.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f9029ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431586 cookie 94802894911328 notifier 6276 bl.length()=885 2024-05-09T20:25:52.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f9029ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 info.flags=0x17 2024-05-09T20:25:52.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f9029ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 to cache LRU end 2024-05-09T20:25:52.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.701+0000 7f9029ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:52.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.705+0000 7f91494cb980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20]): orig_sources= new_sources= 2024-05-09T20:25:52.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.705+0000 7f91494cb980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20]): potential sources added=[] removed=[] 2024-05-09T20:25:52.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.705+0000 7f91494cb980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20]): orig_dests= new_dests= 2024-05-09T20:25:52.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.705+0000 7f91494cb980 20 handle_bi_update(): bucket=:error-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20]): potential dests added=[] removed=[] 2024-05-09T20:25:52.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.713+0000 7f91494cb980 1 execute INFO: reshard of bucket "error-at-set-target-layout" completed successfully 2024-05-09T20:25:52.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.713+0000 7f8ffd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:52.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.721+0000 7f91494cb980 20 remove_watcher() i=0 2024-05-09T20:25:52.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.721+0000 7f91494cb980 2 removed watcher, disabling cache 2024-05-09T20:25:52.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.721+0000 7f91494cb980 20 remove_watcher() i=1 2024-05-09T20:25:52.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.725+0000 7f91494cb980 20 remove_watcher() i=2 2024-05-09T20:25:52.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.725+0000 7f91494cb980 20 remove_watcher() i=3 2024-05-09T20:25:52.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.729+0000 7f91494cb980 20 remove_watcher() i=4 2024-05-09T20:25:52.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.729+0000 7f91494cb980 20 remove_watcher() i=5 2024-05-09T20:25:52.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.733+0000 7f91494cb980 20 remove_watcher() i=6 2024-05-09T20:25:52.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.733+0000 7f91494cb980 20 remove_watcher() i=7 2024-05-09T20:25:52.741 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-set-target-layout 2024-05-09T20:25:52.774 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:52.774 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:52.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 20 realm 2024-05-09T20:25:52.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:52.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:52.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.829+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.833+0000 7f981dca1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:52.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.833+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:52.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.833+0000 7f981dca1980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:52.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.833+0000 7f981dca1980 20 searching for the correct realm 2024-05-09T20:25:52.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:52.837+0000 7f981b7b5640 0 -- 172.21.15.63:0/1831233613 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f98140de910 msgr2=0x7f98140dc440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:53.044 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.053+0000 7f981b7b5640 0 -- 172.21.15.63:0/1831233613 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f98140de910 msgr2=0x7f98140dc440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:53.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:53.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:53.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:53.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:53.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:53.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:53.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:53.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.065+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:53.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:53.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 zone default found 2024-05-09T20:25:53.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 4 Realm: () 2024-05-09T20:25:53.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:53.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:53.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:53.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 zonegroup default 2024-05-09T20:25:53.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:53.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:53.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:53.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 started sync module instance, tier type = 2024-05-09T20:25:53.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.069+0000 7f981dca1980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:53.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.105+0000 7f981dca1980 20 add_watcher() i=0 2024-05-09T20:25:53.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.105+0000 7f981dca1980 20 add_watcher() i=1 2024-05-09T20:25:53.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.105+0000 7f981dca1980 20 add_watcher() i=2 2024-05-09T20:25:53.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.105+0000 7f981dca1980 20 add_watcher() i=3 2024-05-09T20:25:53.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.105+0000 7f981dca1980 20 add_watcher() i=4 2024-05-09T20:25:53.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.105+0000 7f981dca1980 20 add_watcher() i=5 2024-05-09T20:25:53.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.105+0000 7f981dca1980 20 add_watcher() i=6 2024-05-09T20:25:53.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.113+0000 7f981dca1980 20 add_watcher() i=7 2024-05-09T20:25:53.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.113+0000 7f981dca1980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:53.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.117+0000 7f96d1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:53.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.117+0000 7f981dca1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:53.108 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.117+0000 7f981dca1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:53.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.117+0000 7f981dca1980 5 note: GC not initialized 2024-05-09T20:25:53.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.117+0000 7f96d0fe1640 20 reqs_thread_entry: start 2024-05-09T20:25:53.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.117+0000 7f981dca1980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:53.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 Filter name: none 2024-05-09T20:25:53.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f964a7fc640 20 reqs_thread_entry: start 2024-05-09T20:25:53.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:53.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:53.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:53.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.112 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:53.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.121+0000 7f981dca1980 20 rados->read ofs=0 len=0 2024-05-09T20:25:53.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.125+0000 7f981dca1980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:53.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.125+0000 7f981dca1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:53.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.129+0000 7f96d1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:53.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.133+0000 7f981dca1980 20 remove_watcher() i=0 2024-05-09T20:25:53.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.133+0000 7f981dca1980 2 removed watcher, disabling cache 2024-05-09T20:25:53.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.133+0000 7f981dca1980 20 remove_watcher() i=1 2024-05-09T20:25:53.127 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.137+0000 7f981dca1980 20 remove_watcher() i=2 2024-05-09T20:25:53.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.141+0000 7f981dca1980 20 remove_watcher() i=3 2024-05-09T20:25:53.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.145+0000 7f981dca1980 20 remove_watcher() i=4 2024-05-09T20:25:53.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.145+0000 7f981dca1980 20 remove_watcher() i=5 2024-05-09T20:25:53.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.145+0000 7f981dca1980 20 remove_watcher() i=6 2024-05-09T20:25:53.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:53.149+0000 7f981dca1980 20 remove_watcher() i=7 2024-05-09T20:25:53.157 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:53.158 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:53.158 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-set-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.20 num_objects: 20 size_kb: 1 num_shards: 12 2024-05-09T20:25:53.158 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:54.139 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with abort at set_target_layout 2024-05-09T20:25:54.139 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:56.268 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-05-09T20:25:56.302 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:56.302 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:56.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.357+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 realm 2024-05-09T20:25:56.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:56.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:56.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:56.352 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.361+0000 7f6cc943a980 20 searching for the correct realm 2024-05-09T20:25:56.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.365+0000 7f6cc774f640 0 -- 172.21.15.63:0/1478425874 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f6cbc069b80 msgr2=0x7f6cbc072ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:56.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.577+0000 7f6cc674d640 0 -- 172.21.15.63:0/1478425874 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f6cb80e8060 msgr2=0x7f6cb81167e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:56.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.593+0000 7f6cc943a980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:56.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.593+0000 7f6cc943a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:56.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.593+0000 7f6cc943a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:56.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.593+0000 7f6cc943a980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:56.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.593+0000 7f6cc943a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:56.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.593+0000 7f6cc943a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:56.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.593+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:56.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:56.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 zone default found 2024-05-09T20:25:56.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 4 Realm: () 2024-05-09T20:25:56.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:56.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:56.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:56.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 zonegroup default 2024-05-09T20:25:56.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.597+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.601+0000 7f6cc943a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.601+0000 7f6cc943a980 20 started sync module instance, tier type = 2024-05-09T20:25:56.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.601+0000 7f6cc943a980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:56.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.621+0000 7f6cc943a980 20 add_watcher() i=0 2024-05-09T20:25:56.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.621+0000 7f6cc943a980 20 add_watcher() i=1 2024-05-09T20:25:56.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.621+0000 7f6cc943a980 20 add_watcher() i=2 2024-05-09T20:25:56.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.621+0000 7f6cc943a980 20 add_watcher() i=3 2024-05-09T20:25:56.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.621+0000 7f6cc943a980 20 add_watcher() i=4 2024-05-09T20:25:56.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.621+0000 7f6cc943a980 20 add_watcher() i=5 2024-05-09T20:25:56.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.621+0000 7f6cc943a980 20 add_watcher() i=6 2024-05-09T20:25:56.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.625+0000 7f6cc943a980 20 add_watcher() i=7 2024-05-09T20:25:56.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.625+0000 7f6cc943a980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:56.618 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6b7d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:56.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6cc943a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:56.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6cc943a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:56.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6cc943a980 5 note: GC not initialized 2024-05-09T20:25:56.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6af7fff640 20 reqs_thread_entry: start 2024-05-09T20:25:56.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6cc943a980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:56.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6cc943a980 20 Filter name: none 2024-05-09T20:25:56.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6af5ffb640 20 reqs_thread_entry: start 2024-05-09T20:25:56.621 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.629+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:56.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:56.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:56.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.623 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:56.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:56.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.633+0000 7f6cc943a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:56.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.637+0000 7f6b7d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:56.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.641+0000 7f6cc943a980 20 remove_watcher() i=0 2024-05-09T20:25:56.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.641+0000 7f6cc943a980 2 removed watcher, disabling cache 2024-05-09T20:25:56.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.645+0000 7f6cc943a980 20 remove_watcher() i=1 2024-05-09T20:25:56.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.645+0000 7f6cc943a980 20 remove_watcher() i=2 2024-05-09T20:25:56.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.649+0000 7f6cc943a980 20 remove_watcher() i=3 2024-05-09T20:25:56.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.649+0000 7f6cc943a980 20 remove_watcher() i=4 2024-05-09T20:25:56.642 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.653+0000 7f6cc943a980 20 remove_watcher() i=5 2024-05-09T20:25:56.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.653+0000 7f6cc943a980 20 remove_watcher() i=6 2024-05-09T20:25:56.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.657+0000 7f6cc943a980 20 remove_watcher() i=7 2024-05-09T20:25:56.662 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:56.663 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:56.663 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-set-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:25:56.663 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:56.663 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at set_target_layout 2024-05-09T20:25:56.696 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:56.696 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:56.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.749+0000 7f4b33187980 20 realm 2024-05-09T20:25:56.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:56.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:56.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:56.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.753+0000 7f4b33187980 20 searching for the correct realm 2024-05-09T20:25:56.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.757+0000 7f4b30c9b640 0 -- 172.21.15.63:0/1282570987 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f4b28126040 msgr2=0x7f4b281109d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:56.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.973+0000 7f4b30c9b640 0 -- 172.21.15.63:0/1282570987 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f4b28126040 msgr2=0x7f4b281109d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:56.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:56.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:56.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:56.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:56.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:56.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:56.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:56.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:56.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 zone default found 2024-05-09T20:25:56.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 4 Realm: () 2024-05-09T20:25:56.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:56.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:56.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:56.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 zonegroup default 2024-05-09T20:25:56.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.979 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:56.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:56.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 started sync module instance, tier type = 2024-05-09T20:25:56.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:56.989+0000 7f4b33187980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:57.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=0 2024-05-09T20:25:57.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=1 2024-05-09T20:25:57.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=2 2024-05-09T20:25:57.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=3 2024-05-09T20:25:57.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=4 2024-05-09T20:25:57.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=5 2024-05-09T20:25:57.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=6 2024-05-09T20:25:57.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=7 2024-05-09T20:25:57.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.013+0000 7f4b33187980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:57.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.017+0000 7f49e6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:57.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.017+0000 7f4b33187980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:57.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.017+0000 7f4b33187980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:57.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 5 note: GC not initialized 2024-05-09T20:25:57.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f49e5fe3640 20 reqs_thread_entry: start 2024-05-09T20:25:57.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:57.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 20 Filter name: none 2024-05-09T20:25:57.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f49577fe640 20 reqs_thread_entry: start 2024-05-09T20:25:57.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-09T20:25:57.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:57.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-09T20:25:57.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-09T20:25:57.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 : miss 2024-05-09T20:25:57.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:57.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:25:57.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:25:57.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:57.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:57.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.021+0000 7f4b33187980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 2024-05-09T20:25:57.022 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f4b33187980 time 2024-05-09T20:25:57.034696+0000 2024-05-09T20:25:57.022 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:25:57.023 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:25:57.023 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f4b36a5de25] 2024-05-09T20:25:57.023 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f4b36a5dfd9] 2024-05-09T20:25:57.023 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e526391ca1] 2024-05-09T20:25:57.023 INFO:tasks.workunit.client.0.smithi063.stderr: 4: radosgw-admin(+0x90ec3a) [0x55e52639dc3a] 2024-05-09T20:25:57.023 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x253) [0x55e5263a2353] 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr: 6: main() 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4b358c7d90] 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr: 8: __libc_start_main() 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr: 9: _start() 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr:*** Caught signal (Aborted) ** 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f4b33187980 thread_name:radosgw-admin 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.033+0000 7f4b33187980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f4b33187980 time 2024-05-09T20:25:57.034696+0000 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:25:57.024 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f4b36a5de25] 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f4b36a5dfd9] 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e526391ca1] 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 4: radosgw-admin(+0x90ec3a) [0x55e52639dc3a] 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x253) [0x55e5263a2353] 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 6: main() 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4b358c7d90] 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 8: __libc_start_main() 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 9: _start() 2024-05-09T20:25:57.025 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.038 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:25:57.038 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4b358e0520] 2024-05-09T20:25:57.038 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:25:57.038 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:25:57.038 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:25:57.038 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f4b36a5de7f] 2024-05-09T20:25:57.038 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f4b36a5dfd9] 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e526391ca1] 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 8: radosgw-admin(+0x90ec3a) [0x55e52639dc3a] 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x253) [0x55e5263a2353] 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 10: main() 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4b358c7d90] 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 12: __libc_start_main() 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 13: _start() 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.033+0000 7f4b33187980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f4b33187980 thread_name:radosgw-admin 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4b358e0520] 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:25:57.039 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f4b36a5de7f] 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f4b36a5dfd9] 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e526391ca1] 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 8: radosgw-admin(+0x90ec3a) [0x55e52639dc3a] 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x253) [0x55e5263a2353] 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 10: main() 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4b358c7d90] 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 12: __libc_start_main() 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 13: _start() 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr:--- begin dump of recent events --- 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: -499> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): get_auth_request con 0x55e527778b30 auth_method 0 2024-05-09T20:25:57.040 INFO:tasks.workunit.client.0.smithi063.stderr: -498> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -497> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -496> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -495> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -494> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -493> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -492> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -491> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -490> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -489> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_done global_id 6321 payload 274 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -488> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -487> 2024-05-09T20:25:56.729+0000 7f4b3149c640 1 monclient: found mon.noname-a 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -486> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -485> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.041 INFO:tasks.workunit.client.0.smithi063.stderr: -484> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr: -483> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: dump: 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr: -482> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.042 INFO:tasks.workunit.client.0.smithi063.stderr: -481> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr: -480> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr: -479> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr: -478> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr: -477> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: dump: 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr: -476> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 monclient: authenticate success, global_id 6321 2024-05-09T20:25:57.043 INFO:tasks.workunit.client.0.smithi063.stderr: -475> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: get_monmap_and_config success 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr: -474> 2024-05-09T20:25:56.729+0000 7f4b33187980 4 set_mon_vals no callback set 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr: -473> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: shutdown 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr: -472> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 asok(0x55e5276b3430) unregister_commands rotate-key 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr: -471> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: build_initial_monmap 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr: -470> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 build_initial for_mkfs: 0 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr: -469> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: monmap: 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.044 INFO:tasks.workunit.client.0.smithi063.stderr: -468> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding auth protocol: cephx 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -467> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding auth protocol: cephx 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -466> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding auth protocol: cephx 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -465> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -464> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -463> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -462> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -461> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -460> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -459> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -458> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -457> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.045 INFO:tasks.workunit.client.0.smithi063.stderr: -456> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -455> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -454> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -453> 2024-05-09T20:25:56.729+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -452> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 librados: starting msgr at 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -451> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 librados: starting objecter 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -450> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command objecter_requests hook 0x55e5278c2070 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -449> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 librados: setting wanted keys 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -448> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 librados: calling monclient init 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -447> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: init 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -446> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding auth protocol: cephx 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -445> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding auth protocol: cephx 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -444> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding auth protocol: cephx 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -443> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.046 INFO:tasks.workunit.client.0.smithi063.stderr: -442> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -441> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -440> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -439> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -438> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -437> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -436> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -435> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -434> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -433> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -432> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -431> 2024-05-09T20:25:56.729+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -430> 2024-05-09T20:25:56.729+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -429> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command rotate-key hook 0x55e5278bcab8 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -428> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: _reopen_session rank -1 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -427> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: _add_conns ranks=[2,1,0] 2024-05-09T20:25:57.047 INFO:tasks.workunit.client.0.smithi063.stderr: -426> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-c con 0x55e52776c400 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -425> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-b con 0x55e52776ccc0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -424> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-a con 0x55e527778b30 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -423> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -422> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -421> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -420> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): _renew_subs 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -419> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): authenticate will time out at 1963.871180s 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -418> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): get_auth_request con 0x55e527778b30 auth_method 0 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -417> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -416> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -415> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -414> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request con 0x55e52776c400 auth_method 0 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -413> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -412> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -411> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.048 INFO:tasks.workunit.client.0.smithi063.stderr: -410> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -409> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -408> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -407> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -406> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -405> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -404> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): get_auth_request con 0x55e52776ccc0 auth_method 0 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -403> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -402> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -401> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -400> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_done global_id 6324 payload 824 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -399> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -398> 2024-05-09T20:25:56.729+0000 7f4b3149c640 1 monclient: found mon.noname-a 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -397> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -396> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -395> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:25:57.049 INFO:tasks.workunit.client.0.smithi063.stderr: -394> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: dump: 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr: -393> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr: -392> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr: -391> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.050 INFO:tasks.workunit.client.0.smithi063.stderr: -390> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr: -389> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr: -388> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: dump: 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr: -387> 2024-05-09T20:25:56.733+0000 7f4b33187980 5 monclient: authenticate success, global_id 6324 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr: -386> 2024-05-09T20:25:56.733+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr: -385> 2024-05-09T20:25:56.733+0000 7f4a1ee0d640 4 set_mon_vals no callback set 2024-05-09T20:25:57.051 INFO:tasks.workunit.client.0.smithi063.stderr: -384> 2024-05-09T20:25:56.733+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -383> 2024-05-09T20:25:56.733+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -382> 2024-05-09T20:25:56.733+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -381> 2024-05-09T20:25:56.733+0000 7f4b33187980 1 librados: init done 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -380> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -379> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -378> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -377> 2024-05-09T20:25:56.733+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x7f4a18066d60 auth_method 0 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -376> 2024-05-09T20:25:56.733+0000 7f4b3049a640 10 monclient: get_auth_request con 0x55e5278e8240 auth_method 0 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -375> 2024-05-09T20:25:56.733+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x55e527907840 auth_method 0 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -374> 2024-05-09T20:25:56.737+0000 7f4b3049a640 10 monclient: get_auth_request con 0x55e52790ba30 auth_method 0 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -373> 2024-05-09T20:25:56.737+0000 7f4b3149c640 10 monclient: get_auth_request con 0x55e52790b120 auth_method 0 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -372> 2024-05-09T20:25:56.737+0000 7f4b33187980 10 monclient: build_initial_monmap 2024-05-09T20:25:57.052 INFO:tasks.workunit.client.0.smithi063.stderr: -371> 2024-05-09T20:25:56.737+0000 7f4b33187980 1 build_initial for_mkfs: 0 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr: -370> 2024-05-09T20:25:56.737+0000 7f4b33187980 10 monclient: monmap: 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr: -369> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding auth protocol: cephx 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr: -368> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding auth protocol: cephx 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr: -367> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding auth protocol: cephx 2024-05-09T20:25:57.053 INFO:tasks.workunit.client.0.smithi063.stderr: -366> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -365> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -364> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -363> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -362> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -361> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -360> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -359> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -358> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -357> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -356> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -355> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -354> 2024-05-09T20:25:56.737+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -353> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command objecter_requests cmddesc objecter_requests hook 0x55e527918f60 EEXIST 2024-05-09T20:25:57.054 INFO:tasks.workunit.client.0.smithi063.stderr: -352> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: init 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -351> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding auth protocol: cephx 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -350> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding auth protocol: cephx 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -349> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding auth protocol: cephx 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -348> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -347> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -346> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -345> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -344> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -343> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -342> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -341> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -340> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -339> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -338> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -337> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -336> 2024-05-09T20:25:56.741+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.055 INFO:tasks.workunit.client.0.smithi063.stderr: -335> 2024-05-09T20:25:56.741+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -334> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command rotate-key cmddesc rotate-key hook 0x55e5279129c0 EEXIST 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -333> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _reopen_session rank -1 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -332> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -331> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-b con 0x55e52793dbc0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -330> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-a con 0x55e52793dfa0 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -329> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-c con 0x55e527939160 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -328> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -327> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -326> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -325> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): _renew_subs 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -324> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): authenticate will time out at 1963.880570s 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -323> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request con 0x55e527939160 auth_method 0 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -322> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.056 INFO:tasks.workunit.client.0.smithi063.stderr: -321> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -320> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -319> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): get_auth_request con 0x55e52793dfa0 auth_method 0 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -318> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -317> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -316> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -315> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -314> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -313> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -312> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -311> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -310> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -309> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient(hunting): get_auth_request con 0x55e52793dbc0 auth_method 0 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -308> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -307> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.057 INFO:tasks.workunit.client.0.smithi063.stderr: -306> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr: -305> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): handle_auth_done global_id 6327 payload 824 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr: -304> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr: -303> 2024-05-09T20:25:56.741+0000 7f4b3149c640 1 monclient: found mon.noname-a 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr: -302> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr: -301> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr: -300> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr: -299> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: dump: 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.058 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr: -298> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr: -297> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr: -296> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr: -295> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr: -294> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr: -293> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: dump: 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.059 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -292> 2024-05-09T20:25:56.741+0000 7f4b31c9d640 4 set_mon_vals no callback set 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -291> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 monclient: authenticate success, global_id 6327 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -290> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -289> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -288> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -287> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -286> 2024-05-09T20:25:56.741+0000 7f4a167fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -285> 2024-05-09T20:25:56.741+0000 7f4a167fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -284> 2024-05-09T20:25:56.741+0000 7f4a167fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -283> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient: get_auth_request con 0x7f4a08066ca0 auth_method 0 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -282> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: get_monmap_and_config 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -281> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: build_initial_monmap 2024-05-09T20:25:57.060 INFO:tasks.workunit.client.0.smithi063.stderr: -280> 2024-05-09T20:25:56.741+0000 7f4b33187980 1 build_initial for_mkfs: 0 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr: -279> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: monmap: 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr: -278> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr: -277> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.061 INFO:tasks.workunit.client.0.smithi063.stderr: -276> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -275> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -274> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -273> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -272> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -271> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -270> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -269> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -268> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -267> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -266> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -265> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -264> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -263> 2024-05-09T20:25:56.741+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.062 INFO:tasks.workunit.client.0.smithi063.stderr: -262> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: init 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -261> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding auth protocol: cephx 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -260> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding auth protocol: cephx 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -259> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding auth protocol: cephx 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -258> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -257> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -256> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -255> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -254> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -253> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -252> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -251> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -250> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -249> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -248> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.063 INFO:tasks.workunit.client.0.smithi063.stderr: -247> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -246> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -245> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -244> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command rotate-key cmddesc rotate-key hook 0x7ffe8e81e848 EEXIST 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -243> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: _reopen_session rank -1 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -242> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -241> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-b con 0x55e52798f700 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -240> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-a con 0x55e52795d590 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -239> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-c con 0x55e52798b0c0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -238> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -237> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -236> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -235> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): _renew_subs 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -234> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): authenticate will time out at 1963.884482s 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -233> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request con 0x55e52798b0c0 auth_method 0 2024-05-09T20:25:57.064 INFO:tasks.workunit.client.0.smithi063.stderr: -232> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -231> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -230> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -229> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): get_auth_request con 0x55e52795d590 auth_method 0 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -228> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -227> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -226> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -225> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -224> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -223> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -222> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -221> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -220> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -219> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_done global_id 5981 payload 274 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -218> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.065 INFO:tasks.workunit.client.0.smithi063.stderr: -217> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 1 monclient: found mon.noname-c 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr: -216> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr: -215> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr: -214> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr: -213> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: dump: 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.066 INFO:tasks.workunit.client.0.smithi063.stderr: -212> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr: -211> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr: -210> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 monclient: authenticate success, global_id 5981 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr: -209> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr: -208> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: get_monmap_and_config success 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr: -207> 2024-05-09T20:25:56.745+0000 7f4b33187980 4 set_mon_vals no callback set 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr: -206> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr: -205> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr: -204> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: dump: 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.067 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr: -203> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: shutdown 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr: -202> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: build_initial_monmap 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr: -201> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 build_initial for_mkfs: 0 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr: -200> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: monmap: 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.068 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -199> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -198> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -197> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -196> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -195> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -194> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -193> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -192> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -191> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -190> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -189> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.069 INFO:tasks.workunit.client.0.smithi063.stderr: -188> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -187> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -186> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -185> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -184> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -183> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 librados: starting msgr at 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -182> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 librados: starting objecter 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -181> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command objecter_requests cmddesc objecter_requests hook 0x55e527982670 EEXIST 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -180> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 librados: setting wanted keys 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -179> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 librados: calling monclient init 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -178> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: init 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -177> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding auth protocol: cephx 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -176> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding auth protocol: cephx 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -175> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding auth protocol: cephx 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -174> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.070 INFO:tasks.workunit.client.0.smithi063.stderr: -173> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -172> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -171> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -170> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -169> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -168> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -167> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -166> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -165> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -164> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -163> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -162> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -161> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -160> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command rotate-key cmddesc rotate-key hook 0x55e52793b8d8 EEXIST 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -159> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: _reopen_session rank -1 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -158> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:25:57.071 INFO:tasks.workunit.client.0.smithi063.stderr: -157> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-a con 0x55e52795d590 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -156> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-b con 0x55e52798b0c0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -155> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-c con 0x55e52798f700 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -154> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -153> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -152> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -151> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): _renew_subs 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -150> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): authenticate will time out at 1963.887108s 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -149> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): get_auth_request con 0x55e52795d590 auth_method 0 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -148> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -147> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -146> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -145> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request con 0x55e52798f700 auth_method 0 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -144> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -143> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.072 INFO:tasks.workunit.client.0.smithi063.stderr: -142> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -141> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -140> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -139> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -138> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -137> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -136> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -135> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): handle_auth_done global_id 6333 payload 824 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -134> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -133> 2024-05-09T20:25:56.745+0000 7f4b3049a640 1 monclient: found mon.noname-a 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -132> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -131> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient: get_auth_request con 0x55e52798b0c0 auth_method 0 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -130> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -129> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr: -128> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: dump: 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.073 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr: -127> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr: -126> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr: -125> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr: -124> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr: -123> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr: -122> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: dump: 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -121> 2024-05-09T20:25:56.749+0000 7f4b33187980 5 monclient: authenticate success, global_id 6333 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -120> 2024-05-09T20:25:56.745+0000 7f4a07fff640 4 set_mon_vals no callback set 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -119> 2024-05-09T20:25:56.749+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -118> 2024-05-09T20:25:56.749+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -117> 2024-05-09T20:25:56.749+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -116> 2024-05-09T20:25:56.749+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -115> 2024-05-09T20:25:56.749+0000 7f4b33187980 1 librados: init done 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -114> 2024-05-09T20:25:56.749+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cr dump hook 0x55e527987d18 2024-05-09T20:25:57.075 INFO:tasks.workunit.client.0.smithi063.stderr: -113> 2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -112> 2024-05-09T20:25:56.749+0000 7f4a067fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -111> 2024-05-09T20:25:56.749+0000 7f4a067fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -110> 2024-05-09T20:25:56.749+0000 7f4a067fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -109> 2024-05-09T20:25:56.749+0000 7f4b3149c640 10 monclient: get_auth_request con 0x7f49fc066d60 auth_method 0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -108> 2024-05-09T20:25:56.749+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x55e5279a9c40 auth_method 0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -107> 2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -106> 2024-05-09T20:25:56.749+0000 7f4b33187980 20 realm 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -105> 2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -104> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -103> 2024-05-09T20:25:56.753+0000 7f4b33187980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -102> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -101> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -100> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -99> 2024-05-09T20:25:56.753+0000 7f4b3049a640 10 monclient: get_auth_request con 0x55e5279adeb0 auth_method 0 2024-05-09T20:25:57.076 INFO:tasks.workunit.client.0.smithi063.stderr: -98> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -97> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -96> 2024-05-09T20:25:56.753+0000 7f4b3149c640 10 monclient: get_auth_request con 0x55e5279b15d0 auth_method 0 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -95> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -94> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 searching for the correct realm 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -93> 2024-05-09T20:25:56.753+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x55e5279b6390 auth_method 0 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -92> 2024-05-09T20:25:56.757+0000 7f4b3049a640 10 monclient: get_auth_request con 0x7f4b2003ab40 auth_method 0 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -91> 2024-05-09T20:25:56.757+0000 7f4b3149c640 10 monclient: get_auth_request con 0x7f4b1803b060 auth_method 0 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -90> 2024-05-09T20:25:56.757+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x7f4b28126040 auth_method 0 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -89> 2024-05-09T20:25:56.757+0000 7f4b30c9b640 0 -- 172.21.15.63:0/1282570987 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f4b28126040 msgr2=0x7f4b281109d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -88> 2024-05-09T20:25:56.961+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x7f4b28126040 auth_method 0 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -87> 2024-05-09T20:25:56.965+0000 7f4b3049a640 10 monclient: get_auth_request con 0x7f4b2004a040 auth_method 0 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -86> 2024-05-09T20:25:56.973+0000 7f4b30c9b640 0 -- 172.21.15.63:0/1282570987 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f4b28126040 msgr2=0x7f4b281109d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -85> 2024-05-09T20:25:56.973+0000 7f4a067fc640 1 client.6333.objecter ms_handle_reset 0x7f4b28126040 session 0x7f4b280a6070 osd.4 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -84> 2024-05-09T20:25:56.973+0000 7f4a067fc640 10 monclient: _renew_subs 2024-05-09T20:25:57.077 INFO:tasks.workunit.client.0.smithi063.stderr: -83> 2024-05-09T20:25:56.973+0000 7f4a067fc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -82> 2024-05-09T20:25:56.973+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x7f49fc077490 auth_method 0 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -81> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -80> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -79> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -78> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -77> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -76> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -75> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -74> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -73> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -72> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -71> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -70> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:57.078 INFO:tasks.workunit.client.0.smithi063.stderr: -69> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 zone default found 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -68> 2024-05-09T20:25:56.989+0000 7f4b33187980 4 Realm: () 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -67> 2024-05-09T20:25:56.989+0000 7f4b33187980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -66> 2024-05-09T20:25:56.989+0000 7f4b33187980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -65> 2024-05-09T20:25:56.989+0000 7f4b33187980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -64> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 zonegroup default 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -63> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -62> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -61> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -60> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -59> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -58> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -57> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 started sync module instance, tier type = 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -56> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -55> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=0 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -54> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=1 2024-05-09T20:25:57.079 INFO:tasks.workunit.client.0.smithi063.stderr: -53> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=2 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -52> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=3 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -51> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=4 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -50> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=5 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -49> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=6 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -48> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=7 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -47> 2024-05-09T20:25:57.013+0000 7f4b33187980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -46> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding auth protocol: cephx 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -45> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding auth protocol: cephx 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -44> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding auth protocol: cephx 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -43> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -42> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -41> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -40> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -39> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -38> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -37> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.080 INFO:tasks.workunit.client.0.smithi063.stderr: -36> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -35> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -34> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -33> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -32> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -31> 2024-05-09T20:25:57.017+0000 7f49e6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -30> 2024-05-09T20:25:57.017+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -29> 2024-05-09T20:25:57.017+0000 7f4b33187980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -28> 2024-05-09T20:25:57.017+0000 7f4b33187980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -27> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cache list hook 0x55e5279888a0 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -26> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cache inspect hook 0x55e5279888a0 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -25> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cache erase hook 0x55e5279888a0 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -24> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cache zap hook 0x55e5279888a0 2024-05-09T20:25:57.081 INFO:tasks.workunit.client.0.smithi063.stderr: -23> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 note: GC not initialized 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -22> 2024-05-09T20:25:57.021+0000 7f49e5fe3640 20 reqs_thread_entry: start 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -21> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command sync trace show hook 0x7f4b28004470 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -20> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command sync trace history hook 0x7f4b28004470 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -19> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command sync trace active hook 0x7f4b28004470 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -18> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command sync trace active_short hook 0x7f4b28004470 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -17> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -16> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 Filter name: none 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -15> 2024-05-09T20:25:57.021+0000 7f49577fe640 20 reqs_thread_entry: start 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -14> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -13> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -12> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -11> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-09T20:25:57.082 INFO:tasks.workunit.client.0.smithi063.stderr: -10> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -9> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 : miss 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -8> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -7> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -6> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -5> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -4> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -3> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -2> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: -1> 2024-05-09T20:25:57.033+0000 7f4b33187980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f4b33187980 time 2024-05-09T20:25:57.034696+0000 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f4b36a5de25] 2024-05-09T20:25:57.083 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f4b36a5dfd9] 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e526391ca1] 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 4: radosgw-admin(+0x90ec3a) [0x55e52639dc3a] 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x253) [0x55e5263a2353] 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 6: main() 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4b358c7d90] 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 8: __libc_start_main() 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 9: _start() 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 0> 2024-05-09T20:25:57.033+0000 7f4b33187980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f4b33187980 thread_name:radosgw-admin 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4b358e0520] 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:25:57.084 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f4b36a5de7f] 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f4b36a5dfd9] 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e526391ca1] 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 8: radosgw-admin(+0x90ec3a) [0x55e52639dc3a] 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x253) [0x55e5263a2353] 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 10: main() 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4b358c7d90] 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 12: __libc_start_main() 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 13: _start() 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr:--- logging levels --- 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 none 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 lockdep 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 context 2024-05-09T20:25:57.085 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 crush 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_balancer 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_locker 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log_expire 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_migrator 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 3/ 5 mds_quiesce 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 buffer 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 timer 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 filer 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 striper 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 objecter 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rados 2024-05-09T20:25:57.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_mirror 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_replay 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_pwl 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 journaler 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objectcacher 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 immutable_obj_cache 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 client 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 osd 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 optracker 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objclass 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 filestore 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 journal 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 ms 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mon 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/10 monc 2024-05-09T20:25:57.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 paxos 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 tp 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 auth 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 crypto 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 finisher 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 reserver 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 heartbeatmap 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 perfcounter 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 20/20 rgw 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_sync 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_datacache 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_access 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_dbstore 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_flight 2024-05-09T20:25:57.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_lifecycle 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_notification 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 javaclient 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 asok 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 throttle 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 refs 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 compressor 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluestore 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluefs 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 bdev 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 kstore 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 4/ 5 rocksdb 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 fuse 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 2/ 5 mgr 2024-05-09T20:25:57.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mgrc 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 dpdk 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 eventtrace 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 prioritycache 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 test 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephfs_mirror 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephsqlite 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 crimson_interrupt 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_onode 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_odata 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_omap 2024-05-09T20:25:57.090 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_tm 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_t 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cleaner 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_epm 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_lba 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cache 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_journal 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_device 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_backref 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 alienstore 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mclock 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cyanstore 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 ceph_exporter 2024-05-09T20:25:57.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 memstore 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 trace 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 ceph_dedup 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: -2/-2 (syslog threshold) 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 99/99 (stderr threshold) 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 7f49577fe640 / http_manager 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 7f49e5fe3640 / http_manager 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 7f49e6fe5640 / rgw_dt_lg_renew 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a067fc640 / ms_dispatch 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a07fff640 / io_context_pool 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a167fc640 / ms_dispatch 2024-05-09T20:25:57.092 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a1d60a640 / ms_dispatch 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a1ee0d640 / io_context_pool 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b3049a640 / msgr-worker-2 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b30c9b640 / msgr-worker-1 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b3149c640 / msgr-worker-0 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b31c9d640 / io_context_pool 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b33187980 / radosgw-admin 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: max_recent 500 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: max_new 1000 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: log_file /var/log/ceph/ceph-client.admin.79745.log 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr:--- end dump of recent events --- 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr:--- begin dump of recent events --- 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: -499> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): get_auth_request con 0x55e527778b30 auth_method 0 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: -498> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: -497> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.093 INFO:tasks.workunit.client.0.smithi063.stderr: -496> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -495> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -494> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -493> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -492> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -491> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -490> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -489> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_done global_id 6321 payload 274 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -488> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -487> 2024-05-09T20:25:56.729+0000 7f4b3149c640 1 monclient: found mon.noname-a 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -486> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -485> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -484> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:25:57.094 INFO:tasks.workunit.client.0.smithi063.stderr: -483> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: dump: 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr: -482> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr: -481> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr: -480> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.095 INFO:tasks.workunit.client.0.smithi063.stderr: -479> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr: -478> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr: -477> 2024-05-09T20:25:56.729+0000 7f4a1ee0d640 10 monclient: dump: 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.096 INFO:tasks.workunit.client.0.smithi063.stderr: -476> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 monclient: authenticate success, global_id 6321 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr: -475> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: get_monmap_and_config success 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr: -474> 2024-05-09T20:25:56.729+0000 7f4b33187980 4 set_mon_vals no callback set 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr: -473> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: shutdown 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr: -472> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 asok(0x55e5276b3430) unregister_commands rotate-key 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr: -471> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: build_initial_monmap 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr: -470> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 build_initial for_mkfs: 0 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr: -469> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: monmap: 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:25:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -468> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding auth protocol: cephx 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -467> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding auth protocol: cephx 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -466> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding auth protocol: cephx 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -465> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -464> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -463> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -462> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -461> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -460> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -459> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.098 INFO:tasks.workunit.client.0.smithi063.stderr: -458> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -457> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -456> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -455> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: crc 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -454> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278c34c0) adding con mode: secure 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -453> 2024-05-09T20:25:56.729+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -452> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 librados: starting msgr at 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -451> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 librados: starting objecter 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -450> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command objecter_requests hook 0x55e5278c2070 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -449> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 librados: setting wanted keys 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -448> 2024-05-09T20:25:56.729+0000 7f4b33187980 1 librados: calling monclient init 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -447> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: init 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -446> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding auth protocol: cephx 2024-05-09T20:25:57.099 INFO:tasks.workunit.client.0.smithi063.stderr: -445> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding auth protocol: cephx 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -444> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding auth protocol: cephx 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -443> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -442> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -441> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -440> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -439> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -438> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -437> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -436> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -435> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -434> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -433> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: crc 2024-05-09T20:25:57.100 INFO:tasks.workunit.client.0.smithi063.stderr: -432> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 AuthRegistry(0x55e5278bc970) adding con mode: secure 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -431> 2024-05-09T20:25:56.729+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -430> 2024-05-09T20:25:56.729+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -429> 2024-05-09T20:25:56.729+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command rotate-key hook 0x55e5278bcab8 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -428> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: _reopen_session rank -1 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -427> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient: _add_conns ranks=[2,1,0] 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -426> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-c con 0x55e52776c400 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -425> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-b con 0x55e52776ccc0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -424> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-a con 0x55e527778b30 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -423> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -422> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -421> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -420> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): _renew_subs 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -419> 2024-05-09T20:25:56.729+0000 7f4b33187980 10 monclient(hunting): authenticate will time out at 1963.871180s 2024-05-09T20:25:57.101 INFO:tasks.workunit.client.0.smithi063.stderr: -418> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): get_auth_request con 0x55e527778b30 auth_method 0 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -417> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -416> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -415> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -414> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request con 0x55e52776c400 auth_method 0 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -413> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -412> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -411> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -410> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -409> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -408> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -407> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -406> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.102 INFO:tasks.workunit.client.0.smithi063.stderr: -405> 2024-05-09T20:25:56.729+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -404> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): get_auth_request con 0x55e52776ccc0 auth_method 0 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -403> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -402> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -401> 2024-05-09T20:25:56.729+0000 7f4b3049a640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -400> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient(hunting): handle_auth_done global_id 6324 payload 824 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -399> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -398> 2024-05-09T20:25:56.729+0000 7f4b3149c640 1 monclient: found mon.noname-a 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -397> 2024-05-09T20:25:56.729+0000 7f4b3149c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -396> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -395> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr: -394> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: dump: 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.103 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr: -393> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr: -392> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr: -391> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr: -390> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr: -389> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr: -388> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 10 monclient: dump: 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.104 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -387> 2024-05-09T20:25:56.733+0000 7f4b33187980 5 monclient: authenticate success, global_id 6324 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -386> 2024-05-09T20:25:56.733+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -385> 2024-05-09T20:25:56.733+0000 7f4a1ee0d640 4 set_mon_vals no callback set 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -384> 2024-05-09T20:25:56.733+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -383> 2024-05-09T20:25:56.733+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -382> 2024-05-09T20:25:56.733+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -381> 2024-05-09T20:25:56.733+0000 7f4b33187980 1 librados: init done 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -380> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -379> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.105 INFO:tasks.workunit.client.0.smithi063.stderr: -378> 2024-05-09T20:25:56.733+0000 7f4a1d60a640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr: -377> 2024-05-09T20:25:56.733+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x7f4a18066d60 auth_method 0 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr: -376> 2024-05-09T20:25:56.733+0000 7f4b3049a640 10 monclient: get_auth_request con 0x55e5278e8240 auth_method 0 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr: -375> 2024-05-09T20:25:56.733+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x55e527907840 auth_method 0 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr: -374> 2024-05-09T20:25:56.737+0000 7f4b3049a640 10 monclient: get_auth_request con 0x55e52790ba30 auth_method 0 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr: -373> 2024-05-09T20:25:56.737+0000 7f4b3149c640 10 monclient: get_auth_request con 0x55e52790b120 auth_method 0 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr: -372> 2024-05-09T20:25:56.737+0000 7f4b33187980 10 monclient: build_initial_monmap 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr: -371> 2024-05-09T20:25:56.737+0000 7f4b33187980 1 build_initial for_mkfs: 0 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr: -370> 2024-05-09T20:25:56.737+0000 7f4b33187980 10 monclient: monmap: 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:25:57.106 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -369> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding auth protocol: cephx 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -368> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding auth protocol: cephx 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -367> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding auth protocol: cephx 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -366> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -365> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -364> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -363> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -362> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -361> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.107 INFO:tasks.workunit.client.0.smithi063.stderr: -360> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -359> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -358> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -357> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -356> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: crc 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -355> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 AuthRegistry(0x55e5279139f0) adding con mode: secure 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -354> 2024-05-09T20:25:56.737+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -353> 2024-05-09T20:25:56.737+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command objecter_requests cmddesc objecter_requests hook 0x55e527918f60 EEXIST 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -352> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: init 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -351> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding auth protocol: cephx 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -350> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding auth protocol: cephx 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -349> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding auth protocol: cephx 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -348> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -347> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.108 INFO:tasks.workunit.client.0.smithi063.stderr: -346> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -345> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -344> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -343> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -342> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -341> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -340> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -339> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -338> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: crc 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -337> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e527912878) adding con mode: secure 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -336> 2024-05-09T20:25:56.741+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -335> 2024-05-09T20:25:56.741+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -334> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command rotate-key cmddesc rotate-key hook 0x55e5279129c0 EEXIST 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -333> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _reopen_session rank -1 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -332> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -331> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-b con 0x55e52793dbc0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:25:57.109 INFO:tasks.workunit.client.0.smithi063.stderr: -330> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-a con 0x55e52793dfa0 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -329> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-c con 0x55e527939160 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -328> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -327> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -326> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -325> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): _renew_subs 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -324> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient(hunting): authenticate will time out at 1963.880570s 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -323> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request con 0x55e527939160 auth_method 0 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -322> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -321> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -320> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -319> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): get_auth_request con 0x55e52793dfa0 auth_method 0 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -318> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.110 INFO:tasks.workunit.client.0.smithi063.stderr: -317> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -316> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -315> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -314> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -313> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -312> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -311> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -310> 2024-05-09T20:25:56.741+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -309> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient(hunting): get_auth_request con 0x55e52793dbc0 auth_method 0 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -308> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -307> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -306> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -305> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient(hunting): handle_auth_done global_id 6327 payload 824 2024-05-09T20:25:57.111 INFO:tasks.workunit.client.0.smithi063.stderr: -304> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr: -303> 2024-05-09T20:25:56.741+0000 7f4b3149c640 1 monclient: found mon.noname-a 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr: -302> 2024-05-09T20:25:56.741+0000 7f4b3149c640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr: -301> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr: -300> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr: -299> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: dump: 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.112 INFO:tasks.workunit.client.0.smithi063.stderr: -298> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr: -297> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr: -296> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr: -295> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr: -294> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr: -293> 2024-05-09T20:25:56.741+0000 7f4a167fc640 10 monclient: dump: 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.113 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -292> 2024-05-09T20:25:56.741+0000 7f4b31c9d640 4 set_mon_vals no callback set 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -291> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 monclient: authenticate success, global_id 6327 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -290> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -289> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -288> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -287> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -286> 2024-05-09T20:25:56.741+0000 7f4a167fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -285> 2024-05-09T20:25:56.741+0000 7f4a167fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -284> 2024-05-09T20:25:56.741+0000 7f4a167fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -283> 2024-05-09T20:25:56.741+0000 7f4b3049a640 10 monclient: get_auth_request con 0x7f4a08066ca0 auth_method 0 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -282> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: get_monmap_and_config 2024-05-09T20:25:57.114 INFO:tasks.workunit.client.0.smithi063.stderr: -281> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: build_initial_monmap 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr: -280> 2024-05-09T20:25:56.741+0000 7f4b33187980 1 build_initial for_mkfs: 0 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr: -279> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: monmap: 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr: -278> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.115 INFO:tasks.workunit.client.0.smithi063.stderr: -277> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -276> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -275> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -274> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -273> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -272> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -271> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -270> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -269> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -268> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -267> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -266> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -265> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.116 INFO:tasks.workunit.client.0.smithi063.stderr: -264> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -263> 2024-05-09T20:25:56.741+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -262> 2024-05-09T20:25:56.741+0000 7f4b33187980 10 monclient: init 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -261> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding auth protocol: cephx 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -260> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding auth protocol: cephx 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -259> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding auth protocol: cephx 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -258> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -257> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -256> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -255> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -254> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -253> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -252> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -251> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -250> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -249> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.117 INFO:tasks.workunit.client.0.smithi063.stderr: -248> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: crc 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -247> 2024-05-09T20:25:56.741+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81e700) adding con mode: secure 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -246> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -245> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -244> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command rotate-key cmddesc rotate-key hook 0x7ffe8e81e848 EEXIST 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -243> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: _reopen_session rank -1 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -242> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -241> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-b con 0x55e52798f700 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -240> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-a con 0x55e52795d590 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -239> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-c con 0x55e52798b0c0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -238> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -237> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -236> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -235> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): _renew_subs 2024-05-09T20:25:57.118 INFO:tasks.workunit.client.0.smithi063.stderr: -234> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): authenticate will time out at 1963.884482s 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -233> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request con 0x55e52798b0c0 auth_method 0 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -232> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -231> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -230> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -229> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): get_auth_request con 0x55e52795d590 auth_method 0 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -228> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -227> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -226> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -225> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -224> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -223> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -222> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -221> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -220> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -219> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_done global_id 5981 payload 274 2024-05-09T20:25:57.119 INFO:tasks.workunit.client.0.smithi063.stderr: -218> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr: -217> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 1 monclient: found mon.noname-c 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr: -216> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr: -215> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr: -214> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr: -213> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: dump: 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.120 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -212> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -211> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -210> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 monclient: authenticate success, global_id 5981 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -209> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -208> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: get_monmap_and_config success 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -207> 2024-05-09T20:25:56.745+0000 7f4b33187980 4 set_mon_vals no callback set 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -206> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -205> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr: -204> 2024-05-09T20:25:56.745+0000 7f4a07fff640 10 monclient: dump: 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr: -203> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: shutdown 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr: -202> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: build_initial_monmap 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr: -201> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 build_initial for_mkfs: 0 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr: -200> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: monmap: 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:25:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -199> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -198> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -197> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding auth protocol: cephx 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -196> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -195> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -194> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -193> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -192> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -191> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -190> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -189> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -188> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -187> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.123 INFO:tasks.workunit.client.0.smithi063.stderr: -186> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: crc 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -185> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52795afb0) adding con mode: secure 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -184> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -183> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 librados: starting msgr at 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -182> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 librados: starting objecter 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -181> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command objecter_requests cmddesc objecter_requests hook 0x55e527982670 EEXIST 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -180> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 librados: setting wanted keys 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -179> 2024-05-09T20:25:56.745+0000 7f4b33187980 1 librados: calling monclient init 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -178> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: init 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -177> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding auth protocol: cephx 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -176> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding auth protocol: cephx 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -175> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding auth protocol: cephx 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -174> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.124 INFO:tasks.workunit.client.0.smithi063.stderr: -173> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -172> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -171> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -170> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -169> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -168> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -167> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -166> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -165> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -164> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: crc 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -163> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 AuthRegistry(0x55e52793b790) adding con mode: secure 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -162> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -161> 2024-05-09T20:25:56.745+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -160> 2024-05-09T20:25:56.745+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command rotate-key cmddesc rotate-key hook 0x55e52793b8d8 EEXIST 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -159> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: _reopen_session rank -1 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -158> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:25:57.125 INFO:tasks.workunit.client.0.smithi063.stderr: -157> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-a con 0x55e52795d590 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -156> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-b con 0x55e52798b0c0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -155> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): picked mon.noname-c con 0x55e52798f700 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -154> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -153> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -152> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): start opening mon connection 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -151> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): _renew_subs 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -150> 2024-05-09T20:25:56.745+0000 7f4b33187980 10 monclient(hunting): authenticate will time out at 1963.887108s 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -149> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): get_auth_request con 0x55e52795d590 auth_method 0 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -148> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -147> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -146> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -145> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request con 0x55e52798f700 auth_method 0 2024-05-09T20:25:57.126 INFO:tasks.workunit.client.0.smithi063.stderr: -144> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -143> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -142> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -141> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -140> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -139> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -138> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -137> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -136> 2024-05-09T20:25:56.745+0000 7f4b30c9b640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -135> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient(hunting): handle_auth_done global_id 6333 payload 824 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -134> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient: _finish_hunting 0 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -133> 2024-05-09T20:25:56.745+0000 7f4b3049a640 1 monclient: found mon.noname-a 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -132> 2024-05-09T20:25:56.745+0000 7f4b3049a640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -131> 2024-05-09T20:25:56.745+0000 7f4b3149c640 10 monclient: get_auth_request con 0x55e52798b0c0 auth_method 0 2024-05-09T20:25:57.127 INFO:tasks.workunit.client.0.smithi063.stderr: -130> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr: -129> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr: -128> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: dump: 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr: -127> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: _finish_auth 0 2024-05-09T20:25:57.128 INFO:tasks.workunit.client.0.smithi063.stderr: -126> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: _check_auth_tickets 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr: -125> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: handle_config config(0 keys) 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr: -124> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr: -123> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr: -122> 2024-05-09T20:25:56.745+0000 7f4a067fc640 10 monclient: dump: 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:25:57.129 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -121> 2024-05-09T20:25:56.749+0000 7f4b33187980 5 monclient: authenticate success, global_id 6333 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -120> 2024-05-09T20:25:56.745+0000 7f4a07fff640 4 set_mon_vals no callback set 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -119> 2024-05-09T20:25:56.749+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -118> 2024-05-09T20:25:56.749+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -117> 2024-05-09T20:25:56.749+0000 7f4b33187980 10 monclient: _renew_subs 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -116> 2024-05-09T20:25:56.749+0000 7f4b33187980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -115> 2024-05-09T20:25:56.749+0000 7f4b33187980 1 librados: init done 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -114> 2024-05-09T20:25:56.749+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cr dump hook 0x55e527987d18 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -113> 2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -112> 2024-05-09T20:25:56.749+0000 7f4a067fc640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -111> 2024-05-09T20:25:56.749+0000 7f4a067fc640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -110> 2024-05-09T20:25:56.749+0000 7f4a067fc640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -109> 2024-05-09T20:25:56.749+0000 7f4b3149c640 10 monclient: get_auth_request con 0x7f49fc066d60 auth_method 0 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -108> 2024-05-09T20:25:56.749+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x55e5279a9c40 auth_method 0 2024-05-09T20:25:57.130 INFO:tasks.workunit.client.0.smithi063.stderr: -107> 2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -106> 2024-05-09T20:25:56.749+0000 7f4b33187980 20 realm 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -105> 2024-05-09T20:25:56.749+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -104> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -103> 2024-05-09T20:25:56.753+0000 7f4b33187980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -102> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -101> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -100> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -99> 2024-05-09T20:25:56.753+0000 7f4b3049a640 10 monclient: get_auth_request con 0x55e5279adeb0 auth_method 0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -98> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -97> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -96> 2024-05-09T20:25:56.753+0000 7f4b3149c640 10 monclient: get_auth_request con 0x55e5279b15d0 auth_method 0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -95> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -94> 2024-05-09T20:25:56.753+0000 7f4b33187980 20 searching for the correct realm 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -93> 2024-05-09T20:25:56.753+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x55e5279b6390 auth_method 0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -92> 2024-05-09T20:25:56.757+0000 7f4b3049a640 10 monclient: get_auth_request con 0x7f4b2003ab40 auth_method 0 2024-05-09T20:25:57.131 INFO:tasks.workunit.client.0.smithi063.stderr: -91> 2024-05-09T20:25:56.757+0000 7f4b3149c640 10 monclient: get_auth_request con 0x7f4b1803b060 auth_method 0 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -90> 2024-05-09T20:25:56.757+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x7f4b28126040 auth_method 0 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -89> 2024-05-09T20:25:56.757+0000 7f4b30c9b640 0 -- 172.21.15.63:0/1282570987 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f4b28126040 msgr2=0x7f4b281109d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -88> 2024-05-09T20:25:56.961+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x7f4b28126040 auth_method 0 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -87> 2024-05-09T20:25:56.965+0000 7f4b3049a640 10 monclient: get_auth_request con 0x7f4b2004a040 auth_method 0 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -86> 2024-05-09T20:25:56.973+0000 7f4b30c9b640 0 -- 172.21.15.63:0/1282570987 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f4b28126040 msgr2=0x7f4b281109d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -85> 2024-05-09T20:25:56.973+0000 7f4a067fc640 1 client.6333.objecter ms_handle_reset 0x7f4b28126040 session 0x7f4b280a6070 osd.4 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -84> 2024-05-09T20:25:56.973+0000 7f4a067fc640 10 monclient: _renew_subs 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -83> 2024-05-09T20:25:56.973+0000 7f4a067fc640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -82> 2024-05-09T20:25:56.973+0000 7f4b30c9b640 10 monclient: get_auth_request con 0x7f49fc077490 auth_method 0 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -81> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -80> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -79> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -78> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -77> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:57.132 INFO:tasks.workunit.client.0.smithi063.stderr: -76> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -75> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -74> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -73> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -72> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -71> 2024-05-09T20:25:56.985+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -70> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -69> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 zone default found 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -68> 2024-05-09T20:25:56.989+0000 7f4b33187980 4 Realm: () 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -67> 2024-05-09T20:25:56.989+0000 7f4b33187980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -66> 2024-05-09T20:25:56.989+0000 7f4b33187980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -65> 2024-05-09T20:25:56.989+0000 7f4b33187980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -64> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 zonegroup default 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -63> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -62> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -61> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -60> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.133 INFO:tasks.workunit.client.0.smithi063.stderr: -59> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -58> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -57> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 started sync module instance, tier type = 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -56> 2024-05-09T20:25:56.989+0000 7f4b33187980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -55> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=0 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -54> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=1 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -53> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=2 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -52> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=3 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -51> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=4 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -50> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=5 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -49> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=6 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -48> 2024-05-09T20:25:57.013+0000 7f4b33187980 20 add_watcher() i=7 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -47> 2024-05-09T20:25:57.013+0000 7f4b33187980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -46> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding auth protocol: cephx 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -45> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding auth protocol: cephx 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -44> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding auth protocol: cephx 2024-05-09T20:25:57.134 INFO:tasks.workunit.client.0.smithi063.stderr: -43> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -42> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -41> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -40> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -39> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -38> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -37> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -36> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -35> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -34> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -33> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: crc 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -32> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 AuthRegistry(0x7ffe8e81eb90) adding con mode: secure 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -31> 2024-05-09T20:25:57.017+0000 7f49e6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -30> 2024-05-09T20:25:57.017+0000 7f4b33187980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -29> 2024-05-09T20:25:57.017+0000 7f4b33187980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -28> 2024-05-09T20:25:57.017+0000 7f4b33187980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:57.135 INFO:tasks.workunit.client.0.smithi063.stderr: -27> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cache list hook 0x55e5279888a0 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -26> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cache inspect hook 0x55e5279888a0 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -25> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cache erase hook 0x55e5279888a0 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -24> 2024-05-09T20:25:57.017+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command cache zap hook 0x55e5279888a0 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -23> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 note: GC not initialized 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -22> 2024-05-09T20:25:57.021+0000 7f49e5fe3640 20 reqs_thread_entry: start 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -21> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command sync trace show hook 0x7f4b28004470 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -20> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command sync trace history hook 0x7f4b28004470 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -19> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command sync trace active hook 0x7f4b28004470 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -18> 2024-05-09T20:25:57.021+0000 7f4b33187980 5 asok(0x55e5276b3430) register_command sync trace active_short hook 0x7f4b28004470 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -17> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -16> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 Filter name: none 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -15> 2024-05-09T20:25:57.021+0000 7f49577fe640 20 reqs_thread_entry: start 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -14> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-09T20:25:57.136 INFO:tasks.workunit.client.0.smithi063.stderr: -13> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -12> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -11> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -10> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -9> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 : miss 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -8> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -7> 2024-05-09T20:25:57.021+0000 7f4b33187980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -6> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -5> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -4> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -3> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -2> 2024-05-09T20:25:57.021+0000 7f4b33187980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: -1> 2024-05-09T20:25:57.033+0000 7f4b33187980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f4b33187980 time 2024-05-09T20:25:57.034696+0000 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.137 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f4b36a5de25] 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f4b36a5dfd9] 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e526391ca1] 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 4: radosgw-admin(+0x90ec3a) [0x55e52639dc3a] 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x253) [0x55e5263a2353] 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 6: main() 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 7: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4b358c7d90] 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 8: __libc_start_main() 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 9: _start() 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: 0> 2024-05-09T20:25:57.033+0000 7f4b33187980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:25:57.138 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f4b33187980 thread_name:radosgw-admin 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f4b358e0520] 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f4b36a5de7f] 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f4b36a5dfd9] 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e526391ca1] 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 8: radosgw-admin(+0x90ec3a) [0x55e52639dc3a] 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 9: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x253) [0x55e5263a2353] 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 10: main() 2024-05-09T20:25:57.139 INFO:tasks.workunit.client.0.smithi063.stderr: 11: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f4b358c7d90] 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 12: __libc_start_main() 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 13: _start() 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr:--- logging levels --- 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 none 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 lockdep 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 context 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 crush 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_balancer 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_locker 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log_expire 2024-05-09T20:25:57.140 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_migrator 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 3/ 5 mds_quiesce 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 buffer 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 timer 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 filer 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 striper 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 objecter 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rados 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_mirror 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_replay 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_pwl 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 journaler 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objectcacher 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 immutable_obj_cache 2024-05-09T20:25:57.141 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 client 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 osd 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 optracker 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objclass 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 filestore 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 journal 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 ms 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mon 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 0/10 monc 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 paxos 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 tp 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 auth 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 crypto 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 finisher 2024-05-09T20:25:57.142 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 reserver 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 heartbeatmap 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 perfcounter 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 20/20 rgw 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_sync 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_datacache 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_access 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_dbstore 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_flight 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_lifecycle 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_notification 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 javaclient 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 asok 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 throttle 2024-05-09T20:25:57.143 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 refs 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 compressor 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluestore 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluefs 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 bdev 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 kstore 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 4/ 5 rocksdb 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 fuse 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 2/ 5 mgr 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mgrc 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 dpdk 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 eventtrace 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 prioritycache 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 test 2024-05-09T20:25:57.144 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephfs_mirror 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephsqlite 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 crimson_interrupt 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_onode 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_odata 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_omap 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_tm 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_t 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cleaner 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_epm 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_lba 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cache 2024-05-09T20:25:57.145 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_journal 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_device 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_backref 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 alienstore 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mclock 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cyanstore 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 ceph_exporter 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 memstore 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 trace 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 ceph_dedup 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: -2/-2 (syslog threshold) 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 99/99 (stderr threshold) 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 7f49577fe640 / http_manager 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 7f49e5fe3640 / http_manager 2024-05-09T20:25:57.146 INFO:tasks.workunit.client.0.smithi063.stderr: 7f49e6fe5640 / rgw_dt_lg_renew 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a067fc640 / ms_dispatch 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a07fff640 / io_context_pool 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a167fc640 / ms_dispatch 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a1d60a640 / ms_dispatch 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4a1ee0d640 / io_context_pool 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b3049a640 / msgr-worker-2 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b30c9b640 / msgr-worker-1 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b3149c640 / msgr-worker-0 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b31c9d640 / io_context_pool 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: 7f4b33187980 / radosgw-admin 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: max_recent 500 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: max_new 1000 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr: log_file /var/lib/ceph/crash/2024-05-09T20:25:57.037003Z_686bfbd9-9b88-4c79-8ace-ed1cbbc7404a/log 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr:--- end dump of recent events --- 2024-05-09T20:25:57.147 INFO:tasks.workunit.client.0.smithi063.stderr:Aborted 2024-05-09T20:25:57.148 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-05-09T20:25:57.150 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:57.150 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:57.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.121+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 realm 2024-05-09T20:25:57.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.150 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.125+0000 7f83bf966980 20 searching for the correct realm 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.129+0000 7f83bdc7b640 0 -- 172.21.15.63:0/2722524795 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f83b40db020 msgr2=0x7f83b410c9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.137+0000 7f83bdc7b640 0 -- 172.21.15.63:0/2722524795 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f82880774f0 msgr2=0x7f8288079920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:57.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.145+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 zone default found 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 4 Realm: () 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 zonegroup default 2024-05-09T20:25:57.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 started sync module instance, tier type = 2024-05-09T20:25:57.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.149+0000 7f83bf966980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:57.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 20 add_watcher() i=0 2024-05-09T20:25:57.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 20 add_watcher() i=1 2024-05-09T20:25:57.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 20 add_watcher() i=2 2024-05-09T20:25:57.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 20 add_watcher() i=3 2024-05-09T20:25:57.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 20 add_watcher() i=4 2024-05-09T20:25:57.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 20 add_watcher() i=5 2024-05-09T20:25:57.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 20 add_watcher() i=6 2024-05-09T20:25:57.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 20 add_watcher() i=7 2024-05-09T20:25:57.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.165+0000 7f83bf966980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:57.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f82737e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:57.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f83bf966980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:57.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f83bf966980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:57.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f83bf966980 5 note: GC not initialized 2024-05-09T20:25:57.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f82727e4640 20 reqs_thread_entry: start 2024-05-09T20:25:57.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f83bf966980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:57.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f83bf966980 20 Filter name: none 2024-05-09T20:25:57.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f81e3fff640 20 reqs_thread_entry: start 2024-05-09T20:25:57.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.169+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:57.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:57.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:57.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:57.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:57.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.173+0000 7f83bf966980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:25:57.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.177+0000 7f82737e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:57.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.181+0000 7f83bf966980 20 remove_watcher() i=0 2024-05-09T20:25:57.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.181+0000 7f83bf966980 2 removed watcher, disabling cache 2024-05-09T20:25:57.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.181+0000 7f83bf966980 20 remove_watcher() i=1 2024-05-09T20:25:57.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.185+0000 7f83bf966980 20 remove_watcher() i=2 2024-05-09T20:25:57.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.185+0000 7f83bf966980 20 remove_watcher() i=3 2024-05-09T20:25:57.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.189+0000 7f83bf966980 20 remove_watcher() i=4 2024-05-09T20:25:57.179 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.189+0000 7f83bf966980 20 remove_watcher() i=5 2024-05-09T20:25:57.180 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.189+0000 7f83bf966980 20 remove_watcher() i=6 2024-05-09T20:25:57.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.193+0000 7f83bf966980 20 remove_watcher() i=7 2024-05-09T20:25:57.195 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.195 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:25:57.195 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-set-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:25:57.196 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:25:57.291 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:25:57.329 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:25:57.329 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:25:57.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.381+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 realm 2024-05-09T20:25:57.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:25:57.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:57.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.377 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:25:57.377 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.385+0000 7fcb78fa0980 20 searching for the correct realm 2024-05-09T20:25:57.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.389+0000 7fcb762b3640 0 -- 172.21.15.63:0/1241400767 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fca40077440 msgr2=0x7fca40079870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:25:57.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.601+0000 7fcb762b3640 0 -- 172.21.15.63:0/1241400767 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7fcb68015440 msgr2=0x7fcb680158c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:25:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:25:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:25:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:25:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:25:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:25:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:25:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:25:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:25:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 zone default found 2024-05-09T20:25:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 4 Realm: () 2024-05-09T20:25:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:25:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:25:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:25:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 zonegroup default 2024-05-09T20:25:57.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.825+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.829+0000 7fcb78fa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.829+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.829+0000 7fcb78fa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.829+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.829+0000 7fcb78fa0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:25:57.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.829+0000 7fcb78fa0980 20 started sync module instance, tier type = 2024-05-09T20:25:57.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.829+0000 7fcb78fa0980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:25:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.849+0000 7fcb78fa0980 20 add_watcher() i=0 2024-05-09T20:25:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.849+0000 7fcb78fa0980 20 add_watcher() i=1 2024-05-09T20:25:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.849+0000 7fcb78fa0980 20 add_watcher() i=2 2024-05-09T20:25:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.849+0000 7fcb78fa0980 20 add_watcher() i=3 2024-05-09T20:25:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.849+0000 7fcb78fa0980 20 add_watcher() i=4 2024-05-09T20:25:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.849+0000 7fcb78fa0980 20 add_watcher() i=5 2024-05-09T20:25:57.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.849+0000 7fcb78fa0980 20 add_watcher() i=6 2024-05-09T20:25:57.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.853+0000 7fcb78fa0980 20 add_watcher() i=7 2024-05-09T20:25:57.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.853+0000 7fcb78fa0980 2 all 8 watchers are set, enabling cache 2024-05-09T20:25:57.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fca2d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:57.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fcb78fa0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:25:57.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fcb78fa0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:25:57.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fcb78fa0980 5 note: GC not initialized 2024-05-09T20:25:57.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fc9a7fff640 20 reqs_thread_entry: start 2024-05-09T20:25:57.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fcb78fa0980 20 init_complete bucket index max shards: 11 2024-05-09T20:25:57.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fcb78fa0980 20 Filter name: none 2024-05-09T20:25:57.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fc9a5ffb640 20 reqs_thread_entry: start 2024-05-09T20:25:57.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fcb78fa0980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-09T20:25:57.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.857+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:25:57.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-09T20:25:57.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-09T20:25:57.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 : miss 2024-05-09T20:25:57.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 20 rados->read ofs=0 len=0 2024-05-09T20:25:57.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:25:57.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:25:57.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:25:57.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:25:57.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:25:57.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.861+0000 7fcb78fa0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 2024-05-09T20:25:57.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.865+0000 7fcb78fa0980 0 INFO: RGWReshardLock::lock found lock on abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to be held by another RGW process; skipping for now 2024-05-09T20:25:57.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.865+0000 7fca2d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:25:57.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.869+0000 7fcb78fa0980 20 remove_watcher() i=0 2024-05-09T20:25:57.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.869+0000 7fcb78fa0980 2 removed watcher, disabling cache 2024-05-09T20:25:57.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.873+0000 7fcb78fa0980 20 remove_watcher() i=1 2024-05-09T20:25:57.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.873+0000 7fcb78fa0980 20 remove_watcher() i=2 2024-05-09T20:25:57.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.877+0000 7fcb78fa0980 20 remove_watcher() i=3 2024-05-09T20:25:57.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.881+0000 7fcb78fa0980 20 remove_watcher() i=4 2024-05-09T20:25:57.871 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.881+0000 7fcb78fa0980 20 remove_watcher() i=5 2024-05-09T20:25:57.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.881+0000 7fcb78fa0980 20 remove_watcher() i=6 2024-05-09T20:25:57.874 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:25:57.885+0000 7fcb78fa0980 20 remove_watcher() i=7 2024-05-09T20:25:57.893 INFO:tasks.workunit.client.0.smithi063.stderr:waiting 30 seconds for reshard lock to expire... 2024-05-09T20:26:27.925 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-set-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:26:27.961 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:27.961 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:28.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.225+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.225+0000 7f2cb7cd2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.225+0000 7f2cb7cd2980 20 realm 2024-05-09T20:26:28.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.225+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.225+0000 7f2cb7cd2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.225+0000 7f2cb7cd2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:28.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.229+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.229+0000 7f2cb7cd2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.229+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.229+0000 7f2cb7cd2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:28.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.229+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.229+0000 7f2cb7cd2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:28.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.229+0000 7f2cb7cd2980 20 searching for the correct realm 2024-05-09T20:26:28.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.229+0000 7f2cb57e6640 0 -- 172.21.15.63:0/2794691276 >> [v2:172.21.15.63:6824/1568250839,v1:172.21.15.63:6825/1568250839] conn(0x5605aaded420 msgr2=0x5605aadef8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:28.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.441+0000 7f2cb57e6640 0 -- 172.21.15.63:0/2794691276 >> [v2:172.21.15.63:6824/1568250839,v1:172.21.15.63:6825/1568250839] conn(0x5605aaded420 msgr2=0x5605aadef8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:28.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:28.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:28.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:28.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:28.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:28.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:28.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.461+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:28.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:28.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 zone default found 2024-05-09T20:26:28.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 4 Realm: () 2024-05-09T20:26:28.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:28.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:28.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:28.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 zonegroup default 2024-05-09T20:26:28.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 started sync module instance, tier type = 2024-05-09T20:26:28.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.465+0000 7f2cb7cd2980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 20 add_watcher() i=0 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 20 add_watcher() i=1 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 20 add_watcher() i=2 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 20 add_watcher() i=3 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 20 add_watcher() i=4 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 20 add_watcher() i=5 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 20 add_watcher() i=6 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 20 add_watcher() i=7 2024-05-09T20:26:28.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.489+0000 7f2cb7cd2980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:28.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.493+0000 7f2b6b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:28.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:28.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:28.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 5 note: GC not initialized 2024-05-09T20:26:28.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2b6a7e4640 20 reqs_thread_entry: start 2024-05-09T20:26:28.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:28.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 20 Filter name: none 2024-05-09T20:26:28.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2adbfff640 20 reqs_thread_entry: start 2024-05-09T20:26:28.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 cache get: name=default.rgw.meta+root+abort-at-set-target-layout : miss 2024-05-09T20:26:28.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 cache put: name=default.rgw.meta+root+abort-at-set-target-layout info.flags=0x11 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 adding default.rgw.meta+root+abort-at-set-target-layout to cache LRU end 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 : miss 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:28.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.497+0000 7f2cb7cd2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 2024-05-09T20:26:28.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.509+0000 7f2cb7cd2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:26:28.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.509+0000 7f2cb7cd2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:26:28.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.509+0000 7f2cb7cd2980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:28.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.509+0000 7f2cb7cd2980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21, ofs0, ns] 2024-05-09T20:26:28.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.509+0000 7f2b87fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431626 cookie 94582341569968 notifier 6390 bl.length()=917 2024-05-09T20:26:28.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.509+0000 7f2b87fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:26:28.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.509+0000 7f2b87fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:26:28.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.509+0000 7f2b87fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:28.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.513+0000 7f2cb7cd2980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21]): orig_sources= new_sources= 2024-05-09T20:26:28.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.513+0000 7f2cb7cd2980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21]): potential sources added=[] removed=[] 2024-05-09T20:26:28.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.513+0000 7f2cb7cd2980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21]): orig_dests= new_dests= 2024-05-09T20:26:28.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.513+0000 7f2cb7cd2980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21]): potential dests added=[] removed=[] 2024-05-09T20:26:28.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.0 2024-05-09T20:26:28.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.0 2024-05-09T20:26:28.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.1 2024-05-09T20:26:28.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.1 2024-05-09T20:26:28.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.2 2024-05-09T20:26:28.546 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.2 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.3 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.3 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.4 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.4 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.5 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.5 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.6 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.6 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.7 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.7 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.8 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.8 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.9 2024-05-09T20:26:28.547 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.9 2024-05-09T20:26:28.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.10 2024-05-09T20:26:28.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.10 2024-05-09T20:26:28.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.11 2024-05-09T20:26:28.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1.11 2024-05-09T20:26:28.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.0 2024-05-09T20:26:28.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.1 2024-05-09T20:26:28.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.521+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.2 2024-05-09T20:26:28.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.525+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.3 2024-05-09T20:26:28.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.525+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.4 2024-05-09T20:26:28.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.525+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.5 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.525+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.6 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.525+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.7 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.529+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.8 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.529+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.9 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.529+0000 7f2cb7cd2980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21.10 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.537+0000 7f2cb7cd2980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 : hit (requested=0x17, cached=0x17) 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.537+0000 7f2cb7cd2980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.541+0000 7f2cb7cd2980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.541+0000 7f2cb7cd2980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.541+0000 7f2cb7cd2980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.541+0000 7f2cb7cd2980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21, ofs0, ns] 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.541+0000 7f2b98ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431627 cookie 94582341569968 notifier 6390 bl.length()=885 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.541+0000 7f2b98ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 info.flags=0x17 2024-05-09T20:26:28.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.541+0000 7f2b98ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-set-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 to cache LRU end 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.541+0000 7f2b98ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.545+0000 7f2cb7cd2980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21]): orig_sources= new_sources= 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.545+0000 7f2cb7cd2980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21]): potential sources added=[] removed=[] 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.545+0000 7f2cb7cd2980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21]): orig_dests= new_dests= 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.545+0000 7f2cb7cd2980 20 handle_bi_update(): bucket=:abort-at-set-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21]): potential dests added=[] removed=[] 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.549+0000 7f2cb7cd2980 1 execute INFO: reshard of bucket "abort-at-set-target-layout" completed successfully 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.553+0000 7f2b6b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.557+0000 7f2cb7cd2980 20 remove_watcher() i=0 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.557+0000 7f2cb7cd2980 2 removed watcher, disabling cache 2024-05-09T20:26:28.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.561+0000 7f2cb7cd2980 20 remove_watcher() i=1 2024-05-09T20:26:28.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.565+0000 7f2cb7cd2980 20 remove_watcher() i=2 2024-05-09T20:26:28.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.565+0000 7f2cb7cd2980 20 remove_watcher() i=3 2024-05-09T20:26:28.558 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.569+0000 7f2cb7cd2980 20 remove_watcher() i=4 2024-05-09T20:26:28.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.569+0000 7f2cb7cd2980 20 remove_watcher() i=5 2024-05-09T20:26:28.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.569+0000 7f2cb7cd2980 20 remove_watcher() i=6 2024-05-09T20:26:28.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.573+0000 7f2cb7cd2980 20 remove_watcher() i=7 2024-05-09T20:26:28.579 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-set-target-layout 2024-05-09T20:26:28.607 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:28.607 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:28.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.661+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 realm 2024-05-09T20:26:28.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:28.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:28.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:28.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.665+0000 7f42fc48d980 20 searching for the correct realm 2024-05-09T20:26:28.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.669+0000 7f42fa7a2640 0 -- 172.21.15.63:0/2103585935 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f42f0042c20 msgr2=0x7f42f007b430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:28.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.881+0000 7f42f97a0640 0 -- 172.21.15.63:0/2103585935 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f42ec008040 msgr2=0x7f42ec00a450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:28.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:28.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:28.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:28.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:28.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:28.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:28.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:28.891 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:28.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 zone default found 2024-05-09T20:26:28.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 4 Realm: () 2024-05-09T20:26:28.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:28.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:28.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:28.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 zonegroup default 2024-05-09T20:26:28.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.901+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.905+0000 7f42fc48d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.905+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.905+0000 7f42fc48d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:28.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.905+0000 7f42fc48d980 20 started sync module instance, tier type = 2024-05-09T20:26:28.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.905+0000 7f42fc48d980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:28.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.925+0000 7f42fc48d980 20 add_watcher() i=0 2024-05-09T20:26:28.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.925+0000 7f42fc48d980 20 add_watcher() i=1 2024-05-09T20:26:28.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.925+0000 7f42fc48d980 20 add_watcher() i=2 2024-05-09T20:26:28.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.925+0000 7f42fc48d980 20 add_watcher() i=3 2024-05-09T20:26:28.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.925+0000 7f42fc48d980 20 add_watcher() i=4 2024-05-09T20:26:28.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.925+0000 7f42fc48d980 20 add_watcher() i=5 2024-05-09T20:26:28.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.925+0000 7f42fc48d980 20 add_watcher() i=6 2024-05-09T20:26:28.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.929+0000 7f42fc48d980 20 add_watcher() i=7 2024-05-09T20:26:28.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.929+0000 7f42fc48d980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:28.922 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f41ab7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:28.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f42fc48d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:28.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f42fc48d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:28.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f42fc48d980 5 note: GC not initialized 2024-05-09T20:26:28.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f41aa7fc640 20 reqs_thread_entry: start 2024-05-09T20:26:28.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f42fc48d980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:28.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f42fc48d980 20 Filter name: none 2024-05-09T20:26:28.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f411bfff640 20 reqs_thread_entry: start 2024-05-09T20:26:28.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.933+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:26:28.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:26:28.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:28.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 rados_obj.operate() r=0 bl.length=194 2024-05-09T20:26:28.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 rados->read ofs=0 len=0 2024-05-09T20:26:28.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 rados_obj.operate() r=0 bl.length=408 2024-05-09T20:26:28.928 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.937+0000 7f42fc48d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:28.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.941+0000 7f41ab7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:28.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.945+0000 7f42fc48d980 20 remove_watcher() i=0 2024-05-09T20:26:28.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.945+0000 7f42fc48d980 2 removed watcher, disabling cache 2024-05-09T20:26:28.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.949+0000 7f42fc48d980 20 remove_watcher() i=1 2024-05-09T20:26:28.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.953+0000 7f42fc48d980 20 remove_watcher() i=2 2024-05-09T20:26:28.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.953+0000 7f42fc48d980 20 remove_watcher() i=3 2024-05-09T20:26:28.946 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.957+0000 7f42fc48d980 20 remove_watcher() i=4 2024-05-09T20:26:28.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.957+0000 7f42fc48d980 20 remove_watcher() i=5 2024-05-09T20:26:28.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.957+0000 7f42fc48d980 20 remove_watcher() i=6 2024-05-09T20:26:28.950 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:28.961+0000 7f42fc48d980 20 remove_watcher() i=7 2024-05-09T20:26:28.966 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:28.966 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:26:28.966 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-set-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.21 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-09T20:26:28.966 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:29.840 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with EIO injected at block_writes 2024-05-09T20:26:29.841 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:31.905 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-05-09T20:26:31.940 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:31.941 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:31.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:31.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:31.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 20 realm 2024-05-09T20:26:31.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:31.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:31.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:31.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:31.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:31.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:31.997+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:31.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.001+0000 7f9df9175980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:31.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.001+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:31.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.001+0000 7f9df9175980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:31.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.001+0000 7f9df9175980 20 searching for the correct realm 2024-05-09T20:26:31.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.005+0000 7f9df6488640 0 -- 172.21.15.63:0/3270601171 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f9de80d2040 msgr2=0x7f9de80c0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:32.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.217+0000 7f9df6488640 0 -- 172.21.15.63:0/3270601171 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f9de80bf040 msgr2=0x7f9de80e3430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:32.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:32.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:32.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:32.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:32.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:32.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:32.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:32.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:32.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:32.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 zone default found 2024-05-09T20:26:32.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 4 Realm: () 2024-05-09T20:26:32.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:32.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:32.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:32.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 zonegroup default 2024-05-09T20:26:32.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.441+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.445+0000 7f9df9175980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:32.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.445+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.445+0000 7f9df9175980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:32.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.445+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.445+0000 7f9df9175980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:32.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.445+0000 7f9df9175980 20 started sync module instance, tier type = 2024-05-09T20:26:32.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.445+0000 7f9df9175980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:32.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.469+0000 7f9df9175980 20 add_watcher() i=0 2024-05-09T20:26:32.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.469+0000 7f9df9175980 20 add_watcher() i=1 2024-05-09T20:26:32.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.469+0000 7f9df9175980 20 add_watcher() i=2 2024-05-09T20:26:32.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.469+0000 7f9df9175980 20 add_watcher() i=3 2024-05-09T20:26:32.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.469+0000 7f9df9175980 20 add_watcher() i=4 2024-05-09T20:26:32.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.469+0000 7f9df9175980 20 add_watcher() i=5 2024-05-09T20:26:32.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.469+0000 7f9df9175980 20 add_watcher() i=6 2024-05-09T20:26:32.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.473+0000 7f9df9175980 20 add_watcher() i=7 2024-05-09T20:26:32.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.473+0000 7f9df9175980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:32.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.477+0000 7f9cadfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:32.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.477+0000 7f9df9175980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:32.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.477+0000 7f9df9175980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:32.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.477+0000 7f9df9175980 5 note: GC not initialized 2024-05-09T20:26:32.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.477+0000 7f9cacfe1640 20 reqs_thread_entry: start 2024-05-09T20:26:32.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9df9175980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:32.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9df9175980 20 Filter name: none 2024-05-09T20:26:32.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9ca5ffb640 20 reqs_thread_entry: start 2024-05-09T20:26:32.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9df9175980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:32.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9df9175980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:32.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9df9175980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:32.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.481+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.485+0000 7f9df9175980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:32.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.485+0000 7f9df9175980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.485+0000 7f9df9175980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:32.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.485+0000 7f9df9175980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:32.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.489+0000 7f9cadfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:32.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.493+0000 7f9df9175980 20 remove_watcher() i=0 2024-05-09T20:26:32.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.493+0000 7f9df9175980 2 removed watcher, disabling cache 2024-05-09T20:26:32.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.497+0000 7f9df9175980 20 remove_watcher() i=1 2024-05-09T20:26:32.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.497+0000 7f9df9175980 20 remove_watcher() i=2 2024-05-09T20:26:32.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.501+0000 7f9df9175980 20 remove_watcher() i=3 2024-05-09T20:26:32.492 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.501+0000 7f9df9175980 20 remove_watcher() i=4 2024-05-09T20:26:32.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.505+0000 7f9df9175980 20 remove_watcher() i=5 2024-05-09T20:26:32.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.505+0000 7f9df9175980 20 remove_watcher() i=6 2024-05-09T20:26:32.497 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.505+0000 7f9df9175980 20 remove_watcher() i=7 2024-05-09T20:26:32.515 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:32.515 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:26:32.515 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-block-writes id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:26:32.515 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:32.515 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at block_writes 2024-05-09T20:26:32.553 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:32.553 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:32.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:32.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 20 realm 2024-05-09T20:26:32.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:32.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:32.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:32.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.609+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.613+0000 7f66d6074980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:32.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.613+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:32.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.613+0000 7f66d6074980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:32.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.613+0000 7f66d6074980 20 searching for the correct realm 2024-05-09T20:26:32.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:32.617+0000 7f66d3b88640 0 -- 172.21.15.63:0/1333138934 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f66cc055820 msgr2=0x7f66cc07e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:33.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.033+0000 7f66d4389640 0 -- 172.21.15.63:0/1333138934 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f66c8027020 msgr2=0x7f66c8060a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:33.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:33.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:33.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:33.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:33.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:33.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:33.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:33.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:33.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 zone default found 2024-05-09T20:26:33.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 4 Realm: () 2024-05-09T20:26:33.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:33.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:33.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:33.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 zonegroup default 2024-05-09T20:26:33.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.049+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.053+0000 7f66d6074980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.053+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.053+0000 7f66d6074980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.053+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.053+0000 7f66d6074980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.053+0000 7f66d6074980 20 started sync module instance, tier type = 2024-05-09T20:26:33.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.053+0000 7f66d6074980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.077+0000 7f66d6074980 20 add_watcher() i=0 2024-05-09T20:26:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.077+0000 7f66d6074980 20 add_watcher() i=1 2024-05-09T20:26:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.077+0000 7f66d6074980 20 add_watcher() i=2 2024-05-09T20:26:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.077+0000 7f66d6074980 20 add_watcher() i=3 2024-05-09T20:26:33.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.077+0000 7f66d6074980 20 add_watcher() i=4 2024-05-09T20:26:33.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.077+0000 7f66d6074980 20 add_watcher() i=5 2024-05-09T20:26:33.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.077+0000 7f66d6074980 20 add_watcher() i=6 2024-05-09T20:26:33.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.081+0000 7f66d6074980 20 add_watcher() i=7 2024-05-09T20:26:33.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.081+0000 7f66d6074980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:33.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f6589fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:33.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f66d6074980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:33.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f66d6074980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:33.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f66d6074980 5 note: GC not initialized 2024-05-09T20:26:33.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f6588fe1640 20 reqs_thread_entry: start 2024-05-09T20:26:33.077 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f66d6074980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:33.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f66d6074980 20 Filter name: none 2024-05-09T20:26:33.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f65027fc640 20 reqs_thread_entry: start 2024-05-09T20:26:33.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f66d6074980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-05-09T20:26:33.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.085+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:33.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-05-09T20:26:33.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-05-09T20:26:33.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 : miss 2024-05-09T20:26:33.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:33.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:33.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:33.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:26:33.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:33.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.089+0000 7f66d6074980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 2024-05-09T20:26:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22, ofs0, ns] 2024-05-09T20:26:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f65b2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431581 cookie 94907859344688 notifier 6086 bl.length()=907 2024-05-09T20:26:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f65b2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:33.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f65b2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:33.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f65b2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:33.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): orig_sources= new_sources= 2024-05-09T20:26:33.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): potential sources added=[] removed=[] 2024-05-09T20:26:33.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): orig_dests= new_dests= 2024-05-09T20:26:33.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): potential dests added=[] removed=[] 2024-05-09T20:26:33.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 -1 Injecting error=-5 at location=block_writes 2024-05-09T20:26:33.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.101+0000 7f66d6074980 0 ERROR: init_reshard failed to pause writes to the current index: (5) Input/output error 2024-05-09T20:26:33.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.105+0000 7f66d6074980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 : hit (requested=0x17, cached=0x17) 2024-05-09T20:26:33.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.105+0000 7f66d6074980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 2024-05-09T20:26:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.109+0000 7f66d6074980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.109+0000 7f66d6074980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.109+0000 7f66d6074980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.109+0000 7f66d6074980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22, ofs0, ns] 2024-05-09T20:26:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.109+0000 7f65b2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431582 cookie 94907859344688 notifier 6086 bl.length()=875 2024-05-09T20:26:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.109+0000 7f65b2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.109+0000 7f65b2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.109+0000 7f65b2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:33.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.113+0000 7f66d6074980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): orig_sources= new_sources= 2024-05-09T20:26:33.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.113+0000 7f66d6074980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): potential sources added=[] removed=[] 2024-05-09T20:26:33.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.113+0000 7f66d6074980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): orig_dests= new_dests= 2024-05-09T20:26:33.103 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.113+0000 7f66d6074980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): potential dests added=[] removed=[] 2024-05-09T20:26:33.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.117+0000 7f6589fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:33.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.121+0000 7f66d6074980 20 remove_watcher() i=0 2024-05-09T20:26:33.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.121+0000 7f66d6074980 2 removed watcher, disabling cache 2024-05-09T20:26:33.113 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.125+0000 7f66d6074980 20 remove_watcher() i=1 2024-05-09T20:26:33.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.125+0000 7f66d6074980 20 remove_watcher() i=2 2024-05-09T20:26:33.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.129+0000 7f66d6074980 20 remove_watcher() i=3 2024-05-09T20:26:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.129+0000 7f66d6074980 20 remove_watcher() i=4 2024-05-09T20:26:33.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.133+0000 7f66d6074980 20 remove_watcher() i=5 2024-05-09T20:26:33.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.133+0000 7f66d6074980 20 remove_watcher() i=6 2024-05-09T20:26:33.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.137+0000 7f66d6074980 20 remove_watcher() i=7 2024-05-09T20:26:33.141 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-05-09T20:26:33.169 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:33.170 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:33.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 20 realm 2024-05-09T20:26:33.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:33.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.225+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.229+0000 7fe682391980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:33.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.229+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.229+0000 7fe682391980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:33.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.229+0000 7fe682391980 20 searching for the correct realm 2024-05-09T20:26:33.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.233+0000 7fe67fea5640 0 -- 172.21.15.63:0/2037652250 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fe67801d060 msgr2=0x7fe678053b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:33.440 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.449+0000 7fe67fea5640 0 -- 172.21.15.63:0/2037652250 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fe67801d060 msgr2=0x7fe678053b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:33.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:33.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:33.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:33.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:33.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:33.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:33.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.461+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:33.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:33.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 zone default found 2024-05-09T20:26:33.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 4 Realm: () 2024-05-09T20:26:33.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:33.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:33.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:33.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 zonegroup default 2024-05-09T20:26:33.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 started sync module instance, tier type = 2024-05-09T20:26:33.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.465+0000 7fe682391980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:33.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.489+0000 7fe682391980 20 add_watcher() i=0 2024-05-09T20:26:33.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.489+0000 7fe682391980 20 add_watcher() i=1 2024-05-09T20:26:33.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.489+0000 7fe682391980 20 add_watcher() i=2 2024-05-09T20:26:33.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.489+0000 7fe682391980 20 add_watcher() i=3 2024-05-09T20:26:33.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.489+0000 7fe682391980 20 add_watcher() i=4 2024-05-09T20:26:33.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.489+0000 7fe682391980 20 add_watcher() i=5 2024-05-09T20:26:33.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.493+0000 7fe682391980 20 add_watcher() i=6 2024-05-09T20:26:33.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.493+0000 7fe682391980 20 add_watcher() i=7 2024-05-09T20:26:33.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.493+0000 7fe682391980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:33.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.497+0000 7fe535fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:33.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.497+0000 7fe682391980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:33.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.497+0000 7fe682391980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:33.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.497+0000 7fe682391980 5 note: GC not initialized 2024-05-09T20:26:33.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.497+0000 7fe534fe1640 20 reqs_thread_entry: start 2024-05-09T20:26:33.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe682391980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:33.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe682391980 20 Filter name: none 2024-05-09T20:26:33.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe4ae7fc640 20 reqs_thread_entry: start 2024-05-09T20:26:33.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe682391980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:33.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe682391980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:33.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe682391980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:33.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.501+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.505+0000 7fe682391980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:33.494 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.505+0000 7fe682391980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.505+0000 7fe682391980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:33.495 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.505+0000 7fe682391980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:33.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.509+0000 7fe535fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.513+0000 7fe682391980 20 remove_watcher() i=0 2024-05-09T20:26:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.513+0000 7fe682391980 2 removed watcher, disabling cache 2024-05-09T20:26:33.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.513+0000 7fe682391980 20 remove_watcher() i=1 2024-05-09T20:26:33.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.517+0000 7fe682391980 20 remove_watcher() i=2 2024-05-09T20:26:33.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.521+0000 7fe682391980 20 remove_watcher() i=3 2024-05-09T20:26:33.511 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.521+0000 7fe682391980 20 remove_watcher() i=4 2024-05-09T20:26:33.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.521+0000 7fe682391980 20 remove_watcher() i=5 2024-05-09T20:26:33.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.525+0000 7fe682391980 20 remove_watcher() i=6 2024-05-09T20:26:33.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.525+0000 7fe682391980 20 remove_watcher() i=7 2024-05-09T20:26:33.532 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:33.532 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:26:33.532 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-block-writes id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:26:33.532 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:33.643 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:26:33.681 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:33.681 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:33.726 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 20 realm 2024-05-09T20:26:33.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:33.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.728 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.737+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.741+0000 7fe380119980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:33.730 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.741+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.741+0000 7fe380119980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:33.731 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.741+0000 7fe380119980 20 searching for the correct realm 2024-05-09T20:26:33.735 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.745+0000 7fe37dc2d640 0 -- 172.21.15.63:0/489727663 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fe37800e050 msgr2=0x7fe3780552e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:26:33.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.961+0000 7fe37dc2d640 0 -- 172.21.15.63:0/489727663 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fe37800e050 msgr2=0x7fe3780552e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:33.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:33.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:33.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:33.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:33.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:33.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:33.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:33.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.973+0000 7fe380119980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:33.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 zone default found 2024-05-09T20:26:33.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 4 Realm: () 2024-05-09T20:26:33.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:33.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:33.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:33.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 zonegroup default 2024-05-09T20:26:33.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:33.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:33.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 started sync module instance, tier type = 2024-05-09T20:26:33.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:33.977+0000 7fe380119980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:33.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.001+0000 7fe380119980 20 add_watcher() i=0 2024-05-09T20:26:33.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.001+0000 7fe380119980 20 add_watcher() i=1 2024-05-09T20:26:33.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.001+0000 7fe380119980 20 add_watcher() i=2 2024-05-09T20:26:33.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.001+0000 7fe380119980 20 add_watcher() i=3 2024-05-09T20:26:33.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.001+0000 7fe380119980 20 add_watcher() i=4 2024-05-09T20:26:33.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.001+0000 7fe380119980 20 add_watcher() i=5 2024-05-09T20:26:33.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.001+0000 7fe380119980 20 add_watcher() i=6 2024-05-09T20:26:33.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.005+0000 7fe380119980 20 add_watcher() i=7 2024-05-09T20:26:33.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.005+0000 7fe380119980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:33.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.005+0000 7fe2277fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:33.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:33.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:33.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 5 note: GC not initialized 2024-05-09T20:26:33.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe2267fc640 20 reqs_thread_entry: start 2024-05-09T20:26:33.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:34.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 20 Filter name: none 2024-05-09T20:26:34.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe19ffff640 20 reqs_thread_entry: start 2024-05-09T20:26:34.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 10 cache get: name=default.rgw.meta+root+error-at-block-writes : miss 2024-05-09T20:26:34.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:34.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 10 cache put: name=default.rgw.meta+root+error-at-block-writes info.flags=0x11 2024-05-09T20:26:34.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 10 adding default.rgw.meta+root+error-at-block-writes to cache LRU end 2024-05-09T20:26:34.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 : miss 2024-05-09T20:26:34.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.009+0000 7fe380119980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.013+0000 7fe380119980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:34.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.013+0000 7fe380119980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:34.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.013+0000 7fe380119980 10 adding default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:34.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.013+0000 7fe380119980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:26:34.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.013+0000 7fe380119980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:34.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.013+0000 7fe380119980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 2024-05-09T20:26:34.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.021+0000 7fe380119980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:34.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.021+0000 7fe380119980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:34.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.021+0000 7fe380119980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:34.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.021+0000 7fe380119980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22, ofs0, ns] 2024-05-09T20:26:34.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.021+0000 7fe2617fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431583 cookie 94361915232896 notifier 6122 bl.length()=907 2024-05-09T20:26:34.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.021+0000 7fe2617fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:34.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.025+0000 7fe2617fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:34.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.025+0000 7fe2617fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:34.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.025+0000 7fe380119980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): orig_sources= new_sources= 2024-05-09T20:26:34.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.025+0000 7fe380119980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): potential sources added=[] removed=[] 2024-05-09T20:26:34.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.025+0000 7fe380119980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): orig_dests= new_dests= 2024-05-09T20:26:34.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.025+0000 7fe380119980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): potential dests added=[] removed=[] 2024-05-09T20:26:34.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.0 2024-05-09T20:26:34.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.0 2024-05-09T20:26:34.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.1 2024-05-09T20:26:34.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.1 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.2 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.2 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.3 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.3 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.4 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.4 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.5 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.5 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.6 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.6 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.7 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.7 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.8 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.8 2024-05-09T20:26:34.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.9 2024-05-09T20:26:34.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.9 2024-05-09T20:26:34.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.10 2024-05-09T20:26:34.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.10 2024-05-09T20:26:34.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.11 2024-05-09T20:26:34.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1.11 2024-05-09T20:26:34.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.0 2024-05-09T20:26:34.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.1 2024-05-09T20:26:34.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.033+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.2 2024-05-09T20:26:34.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.037+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.3 2024-05-09T20:26:34.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.037+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.4 2024-05-09T20:26:34.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.037+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.5 2024-05-09T20:26:34.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.037+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.6 2024-05-09T20:26:34.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.041+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.7 2024-05-09T20:26:34.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.041+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.8 2024-05-09T20:26:34.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.041+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.9 2024-05-09T20:26:34.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.041+0000 7fe380119980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22.10 2024-05-09T20:26:34.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe380119980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 : hit (requested=0x17, cached=0x17) 2024-05-09T20:26:34.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe380119980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 2024-05-09T20:26:34.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe380119980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:34.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe380119980 10 moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:34.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe380119980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:34.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe380119980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22, ofs0, ns] 2024-05-09T20:26:34.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe2617fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431584 cookie 94361915232896 notifier 6122 bl.length()=875 2024-05-09T20:26:34.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe2617fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 info.flags=0x17 2024-05-09T20:26:34.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe2617fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 to cache LRU end 2024-05-09T20:26:34.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.049+0000 7fe2617fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:34.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.053+0000 7fe380119980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): orig_sources= new_sources= 2024-05-09T20:26:34.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.053+0000 7fe380119980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): potential sources added=[] removed=[] 2024-05-09T20:26:34.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.053+0000 7fe380119980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): orig_dests= new_dests= 2024-05-09T20:26:34.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.053+0000 7fe380119980 20 handle_bi_update(): bucket=:error-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22]): potential dests added=[] removed=[] 2024-05-09T20:26:34.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.057+0000 7fe380119980 1 execute INFO: reshard of bucket "error-at-block-writes" completed successfully 2024-05-09T20:26:34.052 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.061+0000 7fe2277fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:34.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.069+0000 7fe380119980 20 remove_watcher() i=0 2024-05-09T20:26:34.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.069+0000 7fe380119980 2 removed watcher, disabling cache 2024-05-09T20:26:34.060 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.069+0000 7fe380119980 20 remove_watcher() i=1 2024-05-09T20:26:34.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.073+0000 7fe380119980 20 remove_watcher() i=2 2024-05-09T20:26:34.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.073+0000 7fe380119980 20 remove_watcher() i=3 2024-05-09T20:26:34.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.077+0000 7fe380119980 20 remove_watcher() i=4 2024-05-09T20:26:34.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.077+0000 7fe380119980 20 remove_watcher() i=5 2024-05-09T20:26:34.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.081+0000 7fe380119980 20 remove_watcher() i=6 2024-05-09T20:26:34.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.081+0000 7fe380119980 20 remove_watcher() i=7 2024-05-09T20:26:34.088 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-block-writes 2024-05-09T20:26:34.119 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:34.119 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:34.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:34.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 20 realm 2024-05-09T20:26:34.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:34.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:34.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:34.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.173+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.177+0000 7f4d0e586980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:34.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.177+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.177+0000 7f4d0e586980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:34.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.177+0000 7f4d0e586980 20 searching for the correct realm 2024-05-09T20:26:34.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.181+0000 7f4d0c89b640 0 -- 172.21.15.63:0/1629509954 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f4d000c44b0 msgr2=0x7f4d000e7440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:34.177 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.185+0000 7f4d0b899640 0 -- 172.21.15.63:0/1629509954 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f4cfc025c20 msgr2=0x7f4cfc055dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:26:34.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:34.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:34.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:34.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:34.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:34.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:34.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:34.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 zone default found 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 4 Realm: () 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 zonegroup default 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.197+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.201+0000 7f4d0e586980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:34.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.201+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.201+0000 7f4d0e586980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:34.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.201+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.201+0000 7f4d0e586980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:34.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.201+0000 7f4d0e586980 20 started sync module instance, tier type = 2024-05-09T20:26:34.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.201+0000 7f4d0e586980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:34.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 20 add_watcher() i=0 2024-05-09T20:26:34.206 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 20 add_watcher() i=1 2024-05-09T20:26:34.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 20 add_watcher() i=2 2024-05-09T20:26:34.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 20 add_watcher() i=3 2024-05-09T20:26:34.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 20 add_watcher() i=4 2024-05-09T20:26:34.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 20 add_watcher() i=5 2024-05-09T20:26:34.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 20 add_watcher() i=6 2024-05-09T20:26:34.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 20 add_watcher() i=7 2024-05-09T20:26:34.207 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.217+0000 7f4d0e586980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4bc2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 5 note: GC not initialized 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4bc1fe3640 20 reqs_thread_entry: start 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 20 Filter name: none 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4bbaffd640 20 reqs_thread_entry: start 2024-05-09T20:26:34.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:34.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.221+0000 7f4d0e586980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:34.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.225+0000 7f4d0e586980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:34.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.225+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.225+0000 7f4d0e586980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:34.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.225+0000 7f4d0e586980 20 rados->read ofs=0 len=0 2024-05-09T20:26:34.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.225+0000 7f4d0e586980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:34.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.225+0000 7f4d0e586980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:34.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.229+0000 7f4bc2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:34.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.233+0000 7f4d0e586980 20 remove_watcher() i=0 2024-05-09T20:26:34.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.233+0000 7f4d0e586980 2 removed watcher, disabling cache 2024-05-09T20:26:34.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.233+0000 7f4d0e586980 20 remove_watcher() i=1 2024-05-09T20:26:34.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.237+0000 7f4d0e586980 20 remove_watcher() i=2 2024-05-09T20:26:34.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.237+0000 7f4d0e586980 20 remove_watcher() i=3 2024-05-09T20:26:34.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.241+0000 7f4d0e586980 20 remove_watcher() i=4 2024-05-09T20:26:34.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.241+0000 7f4d0e586980 20 remove_watcher() i=5 2024-05-09T20:26:34.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.241+0000 7f4d0e586980 20 remove_watcher() i=6 2024-05-09T20:26:34.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:34.245+0000 7f4d0e586980 20 remove_watcher() i=7 2024-05-09T20:26:34.248 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:34.248 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:26:34.248 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-block-writes id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.22 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-09T20:26:34.248 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:35.191 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with abort at block_writes 2024-05-09T20:26:35.192 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.273 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-05-09T20:26:37.308 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:37.308 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:37.353 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.360+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 realm 2024-05-09T20:26:37.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:37.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.355 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.364+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.368+0000 7f507ffd5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:37.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.368+0000 7f507ffd5980 20 searching for the correct realm 2024-05-09T20:26:37.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.368+0000 7f507e2ea640 0 -- 172.21.15.63:0/2925897549 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f5070008040 msgr2=0x7f5070078e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:26:37.368 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.376+0000 7f507d2e8640 0 -- 172.21.15.63:0/2925897549 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f507403a040 msgr2=0x7f507403c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:37.378 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:37.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:37.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:37.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:37.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:37.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:37.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.380 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 zone default found 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 4 Realm: () 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 zonegroup default 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.381 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.388+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.392+0000 7f507ffd5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.392+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.392+0000 7f507ffd5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.392+0000 7f507ffd5980 20 started sync module instance, tier type = 2024-05-09T20:26:37.382 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.392+0000 7f507ffd5980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:37.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.404+0000 7f507ffd5980 20 add_watcher() i=0 2024-05-09T20:26:37.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f507ffd5980 20 add_watcher() i=1 2024-05-09T20:26:37.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f507ffd5980 20 add_watcher() i=2 2024-05-09T20:26:37.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f507ffd5980 20 add_watcher() i=3 2024-05-09T20:26:37.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f507ffd5980 20 add_watcher() i=4 2024-05-09T20:26:37.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f507ffd5980 20 add_watcher() i=5 2024-05-09T20:26:37.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f507ffd5980 20 add_watcher() i=6 2024-05-09T20:26:37.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f507ffd5980 20 add_watcher() i=7 2024-05-09T20:26:37.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f507ffd5980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:37.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.408+0000 7f4f2f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:37.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:37.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:37.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 5 note: GC not initialized 2024-05-09T20:26:37.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f4f2e7fc640 20 reqs_thread_entry: start 2024-05-09T20:26:37.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:37.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 Filter name: none 2024-05-09T20:26:37.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f4e9ffff640 20 reqs_thread_entry: start 2024-05-09T20:26:37.402 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:37.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:37.403 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:37.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:37.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:37.404 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.412+0000 7f507ffd5980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:37.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.416+0000 7f4f2f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:37.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.420+0000 7f507ffd5980 20 remove_watcher() i=0 2024-05-09T20:26:37.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.420+0000 7f507ffd5980 2 removed watcher, disabling cache 2024-05-09T20:26:37.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.424+0000 7f507ffd5980 20 remove_watcher() i=1 2024-05-09T20:26:37.416 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.424+0000 7f507ffd5980 20 remove_watcher() i=2 2024-05-09T20:26:37.418 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.428+0000 7f507ffd5980 20 remove_watcher() i=3 2024-05-09T20:26:37.420 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.428+0000 7f507ffd5980 20 remove_watcher() i=4 2024-05-09T20:26:37.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.432+0000 7f507ffd5980 20 remove_watcher() i=5 2024-05-09T20:26:37.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.432+0000 7f507ffd5980 20 remove_watcher() i=6 2024-05-09T20:26:37.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.432+0000 7f507ffd5980 20 remove_watcher() i=7 2024-05-09T20:26:37.438 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.439 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:26:37.439 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-block-writes id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:26:37.439 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.439 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at block_writes 2024-05-09T20:26:37.470 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:37.471 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:37.513 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 realm 2024-05-09T20:26:37.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:37.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.515 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.528+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:37.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.528+0000 7faa5bed5980 20 searching for the correct realm 2024-05-09T20:26:37.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.532+0000 7faa599e9640 0 -- 172.21.15.63:0/2733727874 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7faa54038920 msgr2=0x7faa5400f430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:26:37.531 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.540+0000 7faa599e9640 0 -- 172.21.15.63:0/2733727874 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fa9240774c0 msgr2=0x7fa9240798f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:37.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:37.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:37.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:37.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:37.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:37.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:37.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 zone default found 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 4 Realm: () 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 zonegroup default 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 started sync module instance, tier type = 2024-05-09T20:26:37.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.548+0000 7faa5bed5980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:37.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=0 2024-05-09T20:26:37.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=1 2024-05-09T20:26:37.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=2 2024-05-09T20:26:37.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=3 2024-05-09T20:26:37.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=4 2024-05-09T20:26:37.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=5 2024-05-09T20:26:37.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=6 2024-05-09T20:26:37.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 20 add_watcher() i=7 2024-05-09T20:26:37.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:37.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7fa90b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 5 note: GC not initialized 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7fa90a7fc640 20 reqs_thread_entry: start 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 20 Filter name: none 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7fa87bfff640 20 reqs_thread_entry: start 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-09T20:26:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-09T20:26:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-09T20:26:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 : miss 2024-05-09T20:26:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:26:37.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.572+0000 7faa5bed5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 2024-05-09T20:26:37.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.580+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.580+0000 7faa5bed5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.580+0000 7faa5bed5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.580+0000 7faa5bed5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23, ofs0, ns] 2024-05-09T20:26:37.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431631 cookie 94315958939888 notifier 6537 bl.length()=907 2024-05-09T20:26:37.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_sources= new_sources= 2024-05-09T20:26:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential sources added=[] removed=[] 2024-05-09T20:26:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_dests= new_dests= 2024-05-09T20:26:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential dests added=[] removed=[] 2024-05-09T20:26:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7faa5bed5980 time 2024-05-09T20:26:37.588794+0000 2024-05-09T20:26:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:26:37.577 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:26:37.577 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7faa5f7abe25] 2024-05-09T20:26:37.577 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7faa5f7abfd9] 2024-05-09T20:26:37.577 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c7a2c8dca1] 2024-05-09T20:26:37.577 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ad) [0x55c7a2c9e3ad] 2024-05-09T20:26:37.577 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:26:37.577 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7faa5e615d90] 2024-05-09T20:26:37.577 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.584+0000 7faa5bed5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7faa5bed5980 time 2024-05-09T20:26:37.588794+0000 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7faa5f7abe25] 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7faa5f7abfd9] 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c7a2c8dca1] 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ad) [0x55c7a2c9e3ad] 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7faa5e615d90] 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:26:37.578 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:26:37.579 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.579 INFO:tasks.workunit.client.0.smithi063.stderr:*** Caught signal (Aborted) ** 2024-05-09T20:26:37.579 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7faa5bed5980 thread_name:radosgw-admin 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7faa5e62e520] 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7faa5f7abe7f] 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7faa5f7abfd9] 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c7a2c8dca1] 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ad) [0x55c7a2c9e3ad] 2024-05-09T20:26:37.594 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7faa5e615d90] 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.588+0000 7faa5bed5980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7faa5bed5980 thread_name:radosgw-admin 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7faa5e62e520] 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7faa5f7abe7f] 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7faa5f7abfd9] 2024-05-09T20:26:37.595 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c7a2c8dca1] 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ad) [0x55c7a2c9e3ad] 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7faa5e615d90] 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr:--- begin dump of recent events --- 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: -499> 2024-05-09T20:26:37.500+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: -498> 2024-05-09T20:26:37.500+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: -497> 2024-05-09T20:26:37.500+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6164 payload 274 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: -496> 2024-05-09T20:26:37.500+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: -495> 2024-05-09T20:26:37.500+0000 7faa591e8640 1 monclient: found mon.noname-c 2024-05-09T20:26:37.596 INFO:tasks.workunit.client.0.smithi063.stderr: -494> 2024-05-09T20:26:37.500+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr: -493> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr: -492> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr: -491> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: dump: 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr: -490> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr: -489> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.597 INFO:tasks.workunit.client.0.smithi063.stderr: -488> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr: -487> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr: -486> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr: -485> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: dump: 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.598 INFO:tasks.workunit.client.0.smithi063.stderr: -484> 2024-05-09T20:26:37.500+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6164 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr: -483> 2024-05-09T20:26:37.500+0000 7faa5bed5980 10 monclient: get_monmap_and_config success 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr: -482> 2024-05-09T20:26:37.500+0000 7faa5bed5980 4 set_mon_vals no callback set 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr: -481> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: shutdown 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr: -480> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 asok(0x55c7a50a1430) unregister_commands rotate-key 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr: -479> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: build_initial_monmap 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr: -478> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 build_initial for_mkfs: 0 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr: -477> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: monmap: 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.599 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -476> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding auth protocol: cephx 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -475> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding auth protocol: cephx 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -474> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding auth protocol: cephx 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -473> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -472> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -471> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -470> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -469> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -468> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -467> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -466> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.600 INFO:tasks.workunit.client.0.smithi063.stderr: -465> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -464> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -463> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -462> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -461> 2024-05-09T20:26:37.504+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -460> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: starting msgr at 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -459> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: starting objecter 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -458> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command objecter_requests hook 0x55c7a52b3a90 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -457> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: setting wanted keys 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -456> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: calling monclient init 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -455> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: init 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -454> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding auth protocol: cephx 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -453> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding auth protocol: cephx 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -452> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding auth protocol: cephx 2024-05-09T20:26:37.601 INFO:tasks.workunit.client.0.smithi063.stderr: -451> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -450> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -449> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -448> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -447> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -446> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -445> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -444> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -443> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -442> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -441> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -440> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -439> 2024-05-09T20:26:37.504+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -438> 2024-05-09T20:26:37.504+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.602 INFO:tasks.workunit.client.0.smithi063.stderr: -437> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command rotate-key hook 0x55c7a52ad168 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -436> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _reopen_session rank -1 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -435> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _add_conns ranks=[0,2,1] 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -434> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-a con 0x55c7a5138fb0 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -433> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-c con 0x55c7a515a0f0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -432> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-b con 0x55c7a51655a0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -431> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -430> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -429> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -428> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): _renew_subs 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -427> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): authenticate will time out at 2004.645745s 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -426> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): get_auth_request con 0x55c7a515a0f0 auth_method 0 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -425> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -424> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.603 INFO:tasks.workunit.client.0.smithi063.stderr: -423> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -422> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): get_auth_request con 0x55c7a5138fb0 auth_method 0 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -421> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -420> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -419> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -418> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -417> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -416> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -415> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -414> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -413> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -412> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6167 payload 824 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -411> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -410> 2024-05-09T20:26:37.504+0000 7faa591e8640 1 monclient: found mon.noname-c 2024-05-09T20:26:37.604 INFO:tasks.workunit.client.0.smithi063.stderr: -409> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr: -408> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr: -407> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr: -406> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: dump: 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr: -405> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.605 INFO:tasks.workunit.client.0.smithi063.stderr: -404> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr: -403> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr: -402> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr: -401> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr: -400> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: dump: 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.606 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -399> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6167 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -398> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -397> 2024-05-09T20:26:37.504+0000 7fa947e07640 4 set_mon_vals no callback set 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -396> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -395> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -394> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -393> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: init done 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -392> 2024-05-09T20:26:37.508+0000 7fa946604640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -391> 2024-05-09T20:26:37.508+0000 7fa946604640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -390> 2024-05-09T20:26:37.508+0000 7fa946604640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -389> 2024-05-09T20:26:37.508+0000 7faa599e9640 10 monclient: get_auth_request con 0x7fa940066d60 auth_method 0 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -388> 2024-05-09T20:26:37.508+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x55c7a52d7490 auth_method 0 2024-05-09T20:26:37.607 INFO:tasks.workunit.client.0.smithi063.stderr: -387> 2024-05-09T20:26:37.508+0000 7faa599e9640 10 monclient: get_auth_request con 0x55c7a52f79e0 auth_method 0 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr: -386> 2024-05-09T20:26:37.508+0000 7faa591e8640 10 monclient: get_auth_request con 0x55c7a52fbbd0 auth_method 0 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr: -385> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x55c7a52ff8b0 auth_method 0 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr: -384> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: build_initial_monmap 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr: -383> 2024-05-09T20:26:37.512+0000 7faa5bed5980 1 build_initial for_mkfs: 0 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr: -382> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: monmap: 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:26:37.608 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -381> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding auth protocol: cephx 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -380> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding auth protocol: cephx 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -379> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding auth protocol: cephx 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -378> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -377> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -376> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -375> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -374> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -373> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -372> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -371> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -370> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -369> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.609 INFO:tasks.workunit.client.0.smithi063.stderr: -368> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -367> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -366> 2024-05-09T20:26:37.512+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -365> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command objecter_requests cmddesc objecter_requests hook 0x55c7a5308ed0 EEXIST 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -364> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: init 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -363> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding auth protocol: cephx 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -362> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding auth protocol: cephx 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -361> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding auth protocol: cephx 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -360> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -359> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -358> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -357> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -356> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -355> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.610 INFO:tasks.workunit.client.0.smithi063.stderr: -354> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -353> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -352> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -351> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -350> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -349> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -348> 2024-05-09T20:26:37.512+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -347> 2024-05-09T20:26:37.512+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -346> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command rotate-key cmddesc rotate-key hook 0x55c7a5302930 EEXIST 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -345> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _reopen_session rank -1 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -344> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -343> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-a con 0x55c7a532db30 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -342> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-b con 0x55c7a532df10 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -341> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-c con 0x55c7a53290d0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:26:37.611 INFO:tasks.workunit.client.0.smithi063.stderr: -340> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -339> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -338> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -337> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): _renew_subs 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -336> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): authenticate will time out at 2004.655014s 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -335> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): get_auth_request con 0x55c7a532db30 auth_method 0 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -334> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -333> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -332> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -331> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): get_auth_request con 0x55c7a53290d0 auth_method 0 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -330> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -329> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -328> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -327> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.612 INFO:tasks.workunit.client.0.smithi063.stderr: -326> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -325> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -324> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -323> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -322> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -321> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient(hunting): get_auth_request con 0x55c7a532df10 auth_method 0 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -320> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -319> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -318> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -317> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6531 payload 824 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -316> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -315> 2024-05-09T20:26:37.512+0000 7faa591e8640 1 monclient: found mon.noname-a 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -314> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -313> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.613 INFO:tasks.workunit.client.0.smithi063.stderr: -312> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr: -311> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: dump: 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr: -310> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.614 INFO:tasks.workunit.client.0.smithi063.stderr: -309> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr: -308> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6531 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr: -307> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr: -306> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr: -305> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr: -304> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr: -303> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr: -302> 2024-05-09T20:26:37.512+0000 7faa5a9eb640 4 set_mon_vals no callback set 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr: -301> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: dump: 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.615 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -300> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -299> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -298> 2024-05-09T20:26:37.516+0000 7fa93f7fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -297> 2024-05-09T20:26:37.516+0000 7fa93f7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -296> 2024-05-09T20:26:37.516+0000 7fa93f7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -295> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x7fa934066ca0 auth_method 0 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -294> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: get_monmap_and_config 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -293> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: build_initial_monmap 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -292> 2024-05-09T20:26:37.516+0000 7faa5bed5980 1 build_initial for_mkfs: 0 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr: -291> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: monmap: 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:26:37.616 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr: -290> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding auth protocol: cephx 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr: -289> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding auth protocol: cephx 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr: -288> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding auth protocol: cephx 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr: -287> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr: -286> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.617 INFO:tasks.workunit.client.0.smithi063.stderr: -285> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -284> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -283> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -282> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -281> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -280> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -279> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -278> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -277> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -276> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -275> 2024-05-09T20:26:37.516+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -274> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: init 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -273> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding auth protocol: cephx 2024-05-09T20:26:37.618 INFO:tasks.workunit.client.0.smithi063.stderr: -272> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding auth protocol: cephx 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -271> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding auth protocol: cephx 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -270> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -269> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -268> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -267> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -266> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -265> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -264> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -263> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -262> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -261> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -260> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.619 INFO:tasks.workunit.client.0.smithi063.stderr: -259> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -258> 2024-05-09T20:26:37.516+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -257> 2024-05-09T20:26:37.516+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -256> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command rotate-key cmddesc rotate-key hook 0x7fffa3d25c48 EEXIST 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -255> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: _reopen_session rank -1 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -254> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: _add_conns ranks=[0,2,1] 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -253> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-a con 0x55c7a537f720 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -252> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-c con 0x55c7a537fb00 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -251> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-b con 0x55c7a5379990 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -250> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -249> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -248> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -247> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): _renew_subs 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -246> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): authenticate will time out at 2004.658870s 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -245> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): get_auth_request con 0x55c7a537f720 auth_method 0 2024-05-09T20:26:37.620 INFO:tasks.workunit.client.0.smithi063.stderr: -244> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -243> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -242> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -241> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): get_auth_request con 0x55c7a537fb00 auth_method 0 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -240> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -239> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -238> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -237> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -236> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -235> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -234> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -233> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -232> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -231> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6534 payload 274 2024-05-09T20:26:37.621 INFO:tasks.workunit.client.0.smithi063.stderr: -230> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr: -229> 2024-05-09T20:26:37.516+0000 7faa591e8640 1 monclient: found mon.noname-a 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr: -228> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr: -227> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr: -226> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr: -225> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: dump: 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.622 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr: -224> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr: -223> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr: -222> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr: -221> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr: -220> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr: -219> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: dump: 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.623 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr: -218> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6534 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr: -217> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: get_monmap_and_config success 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr: -216> 2024-05-09T20:26:37.516+0000 7faa5bed5980 4 set_mon_vals no callback set 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr: -215> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: shutdown 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr: -214> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: build_initial_monmap 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr: -213> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 build_initial for_mkfs: 0 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr: -212> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: monmap: 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:26:37.624 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -211> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding auth protocol: cephx 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -210> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding auth protocol: cephx 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -209> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding auth protocol: cephx 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -208> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -207> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -206> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -205> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -204> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -203> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -202> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -201> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.625 INFO:tasks.workunit.client.0.smithi063.stderr: -200> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -199> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -198> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -197> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -196> 2024-05-09T20:26:37.520+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -195> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: starting msgr at 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -194> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: starting objecter 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -193> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command objecter_requests cmddesc objecter_requests hook 0x55c7a5328820 EEXIST 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -192> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: setting wanted keys 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -191> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: calling monclient init 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -190> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: init 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -189> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding auth protocol: cephx 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -188> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding auth protocol: cephx 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -187> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding auth protocol: cephx 2024-05-09T20:26:37.626 INFO:tasks.workunit.client.0.smithi063.stderr: -186> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -185> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -184> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -183> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -182> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -181> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -180> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -179> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -178> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -177> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -176> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -175> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -174> 2024-05-09T20:26:37.520+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -173> 2024-05-09T20:26:37.520+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -172> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command rotate-key cmddesc rotate-key hook 0x55c7a534a998 EEXIST 2024-05-09T20:26:37.627 INFO:tasks.workunit.client.0.smithi063.stderr: -171> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _reopen_session rank -1 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -170> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -169> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-a con 0x55c7a537f720 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -168> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-b con 0x55c7a537d100 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -167> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-c con 0x55c7a538c4b0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -166> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -165> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -164> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -163> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): _renew_subs 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -162> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): authenticate will time out at 2004.661443s 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -161> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): get_auth_request con 0x55c7a537f720 auth_method 0 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -160> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -159> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -158> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -157> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): get_auth_request con 0x55c7a538c4b0 auth_method 0 2024-05-09T20:26:37.628 INFO:tasks.workunit.client.0.smithi063.stderr: -156> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -155> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -154> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -153> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -152> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -151> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -150> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -149> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -148> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -147> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6537 payload 824 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -146> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -145> 2024-05-09T20:26:37.520+0000 7faa591e8640 1 monclient: found mon.noname-a 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -144> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -143> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.629 INFO:tasks.workunit.client.0.smithi063.stderr: -142> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr: -141> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: dump: 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr: -140> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.630 INFO:tasks.workunit.client.0.smithi063.stderr: -139> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr: -138> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr: -137> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr: -136> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr: -135> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: dump: 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr: -134> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6537 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr: -133> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.631 INFO:tasks.workunit.client.0.smithi063.stderr: -132> 2024-05-09T20:26:37.520+0000 7fa93d7fa640 4 set_mon_vals no callback set 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -131> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -130> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -129> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -128> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: init done 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -127> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cr dump hook 0x55c7a5384608 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -126> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -125> 2024-05-09T20:26:37.524+0000 7fa92b7fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -124> 2024-05-09T20:26:37.524+0000 7fa92b7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -123> 2024-05-09T20:26:37.524+0000 7fa92b7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -122> 2024-05-09T20:26:37.524+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x7fa924066d90 auth_method 0 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -121> 2024-05-09T20:26:37.524+0000 7faa599e9640 10 monclient: get_auth_request con 0x55c7a5398f70 auth_method 0 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -120> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -119> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 realm 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -118> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -117> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.632 INFO:tasks.workunit.client.0.smithi063.stderr: -116> 2024-05-09T20:26:37.524+0000 7faa5bed5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -115> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -114> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -113> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -112> 2024-05-09T20:26:37.524+0000 7faa591e8640 10 monclient: get_auth_request con 0x55c7a539ce30 auth_method 0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -111> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -110> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -109> 2024-05-09T20:26:37.524+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x55c7a53a06b0 auth_method 0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -108> 2024-05-09T20:26:37.528+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -107> 2024-05-09T20:26:37.528+0000 7faa5bed5980 20 searching for the correct realm 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -106> 2024-05-09T20:26:37.528+0000 7faa599e9640 10 monclient: get_auth_request con 0x55c7a53a5490 auth_method 0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -105> 2024-05-09T20:26:37.528+0000 7faa591e8640 10 monclient: get_auth_request con 0x7faa4c0313f0 auth_method 0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -104> 2024-05-09T20:26:37.528+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x7faa500c8020 auth_method 0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -103> 2024-05-09T20:26:37.532+0000 7faa599e9640 10 monclient: get_auth_request con 0x7faa54038920 auth_method 0 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -102> 2024-05-09T20:26:37.532+0000 7faa599e9640 0 -- 172.21.15.63:0/2733727874 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7faa54038920 msgr2=0x7faa5400f430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -101> 2024-05-09T20:26:37.532+0000 7fa92b7fe640 1 client.6537.objecter ms_handle_reset 0x7faa54038920 session 0x7faa54049040 osd.4 2024-05-09T20:26:37.633 INFO:tasks.workunit.client.0.smithi063.stderr: -100> 2024-05-09T20:26:37.532+0000 7fa92b7fe640 10 monclient: _renew_subs 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -99> 2024-05-09T20:26:37.532+0000 7fa92b7fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -98> 2024-05-09T20:26:37.532+0000 7faa599e9640 10 monclient: get_auth_request con 0x7fa9240774c0 auth_method 0 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -97> 2024-05-09T20:26:37.536+0000 7faa591e8640 10 monclient: get_auth_request con 0x7faa4c02f3e0 auth_method 0 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -96> 2024-05-09T20:26:37.540+0000 7faa599e9640 0 -- 172.21.15.63:0/2733727874 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fa9240774c0 msgr2=0x7fa9240798f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -95> 2024-05-09T20:26:37.540+0000 7fa92b7fe640 1 client.6537.objecter ms_handle_reset 0x7fa9240774c0 session 0x7faa54049040 osd.4 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -94> 2024-05-09T20:26:37.540+0000 7faa599e9640 10 monclient: get_auth_request con 0x7faa54038920 auth_method 0 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -93> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -92> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -91> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -90> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -89> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -88> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:37.634 INFO:tasks.workunit.client.0.smithi063.stderr: -87> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -86> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -85> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -84> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -83> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -82> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -81> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 zone default found 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -80> 2024-05-09T20:26:37.548+0000 7faa5bed5980 4 Realm: () 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -79> 2024-05-09T20:26:37.548+0000 7faa5bed5980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -78> 2024-05-09T20:26:37.548+0000 7faa5bed5980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -77> 2024-05-09T20:26:37.548+0000 7faa5bed5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -76> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 zonegroup default 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -75> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -74> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -73> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -72> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.635 INFO:tasks.workunit.client.0.smithi063.stderr: -71> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -70> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -69> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 started sync module instance, tier type = 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -68> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -67> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=0 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -66> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=1 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -65> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=2 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -64> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=3 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -63> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=4 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -62> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=5 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -61> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=6 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -60> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 add_watcher() i=7 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -59> 2024-05-09T20:26:37.568+0000 7faa5bed5980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -58> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding auth protocol: cephx 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -57> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding auth protocol: cephx 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -56> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding auth protocol: cephx 2024-05-09T20:26:37.636 INFO:tasks.workunit.client.0.smithi063.stderr: -55> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -54> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -53> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -52> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -51> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -50> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -49> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -48> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -47> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -46> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -45> 2024-05-09T20:26:37.568+0000 7fa90b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -44> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -43> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -42> 2024-05-09T20:26:37.568+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -41> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -40> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:37.637 INFO:tasks.workunit.client.0.smithi063.stderr: -39> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cache list hook 0x55c7a5351a80 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -38> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cache inspect hook 0x55c7a5351a80 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -37> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cache erase hook 0x55c7a5351a80 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -36> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cache zap hook 0x55c7a5351a80 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -35> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 note: GC not initialized 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -34> 2024-05-09T20:26:37.568+0000 7fa90a7fc640 20 reqs_thread_entry: start 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -33> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command sync trace show hook 0x7faa500043b0 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -32> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command sync trace history hook 0x7faa500043b0 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -31> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command sync trace active hook 0x7faa500043b0 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -30> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command sync trace active_short hook 0x7faa500043b0 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -29> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -28> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 Filter name: none 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -27> 2024-05-09T20:26:37.568+0000 7fa87bfff640 20 reqs_thread_entry: start 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -26> 2024-05-09T20:26:37.568+0000 7faa5bed5980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-09T20:26:37.638 INFO:tasks.workunit.client.0.smithi063.stderr: -25> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -24> 2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -23> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -22> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -21> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 : miss 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -20> 2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -19> 2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -18> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -17> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -16> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -15> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -14> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -13> 2024-05-09T20:26:37.580+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -12> 2024-05-09T20:26:37.580+0000 7faa5bed5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.639 INFO:tasks.workunit.client.0.smithi063.stderr: -11> 2024-05-09T20:26:37.580+0000 7faa5bed5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -10> 2024-05-09T20:26:37.580+0000 7faa5bed5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23, ofs0, ns] 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -9> 2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431631 cookie 94315958939888 notifier 6537 bl.length()=907 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -8> 2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -7> 2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -6> 2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -5> 2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_sources= new_sources= 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -4> 2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential sources added=[] removed=[] 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -3> 2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_dests= new_dests= 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -2> 2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential dests added=[] removed=[] 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: -1> 2024-05-09T20:26:37.584+0000 7faa5bed5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7faa5bed5980 time 2024-05-09T20:26:37.588794+0000 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:26:37.640 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7faa5f7abe25] 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7faa5f7abfd9] 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c7a2c8dca1] 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ad) [0x55c7a2c9e3ad] 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7faa5e615d90] 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 0> 2024-05-09T20:26:37.588+0000 7faa5bed5980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7faa5bed5980 thread_name:radosgw-admin 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7faa5e62e520] 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:26:37.641 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7faa5f7abe7f] 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7faa5f7abfd9] 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c7a2c8dca1] 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ad) [0x55c7a2c9e3ad] 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7faa5e615d90] 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr:--- logging levels --- 2024-05-09T20:26:37.642 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 none 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 lockdep 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 context 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 crush 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_balancer 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_locker 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log_expire 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_migrator 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 3/ 5 mds_quiesce 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 buffer 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 timer 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 filer 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 striper 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 objecter 2024-05-09T20:26:37.643 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rados 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_mirror 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_replay 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_pwl 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 journaler 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objectcacher 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 immutable_obj_cache 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 client 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 osd 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 optracker 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objclass 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 filestore 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 journal 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 ms 2024-05-09T20:26:37.644 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mon 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 0/10 monc 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 paxos 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 tp 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 auth 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 crypto 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 finisher 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 reserver 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 heartbeatmap 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 perfcounter 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 20/20 rgw 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_sync 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_datacache 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_access 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_dbstore 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_flight 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_lifecycle 2024-05-09T20:26:37.645 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_notification 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 javaclient 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 asok 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 throttle 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 refs 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 compressor 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluestore 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluefs 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 bdev 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 kstore 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 4/ 5 rocksdb 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 fuse 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 2/ 5 mgr 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mgrc 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 dpdk 2024-05-09T20:26:37.646 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 eventtrace 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 prioritycache 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 test 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephfs_mirror 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephsqlite 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 crimson_interrupt 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_onode 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_odata 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_omap 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_tm 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_t 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cleaner 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_epm 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_lba 2024-05-09T20:26:37.647 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cache 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_journal 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_device 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_backref 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 alienstore 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mclock 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cyanstore 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 ceph_exporter 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 memstore 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 trace 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 ceph_dedup 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: -2/-2 (syslog threshold) 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 99/99 (stderr threshold) 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-09T20:26:37.648 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa87bfff640 / http_manager 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa90a7fc640 / http_manager 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa90b7fe640 / rgw_dt_lg_renew 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa92b7fe640 / ms_dispatch 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa93cff9640 / io_context_pool 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa93d7fa640 / io_context_pool 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa93f7fe640 / ms_dispatch 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa946604640 / ms_dispatch 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa947e07640 / io_context_pool 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa591e8640 / msgr-worker-2 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa599e9640 / msgr-worker-1 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa5a1ea640 / msgr-worker-0 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa5a9eb640 / io_context_pool 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa5bed5980 / radosgw-admin 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: max_recent 500 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: max_new 1000 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr: log_file /var/log/ceph/ceph-client.admin.86235.log 2024-05-09T20:26:37.649 INFO:tasks.workunit.client.0.smithi063.stderr:--- end dump of recent events --- 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr:--- begin dump of recent events --- 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -499> 2024-05-09T20:26:37.500+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -498> 2024-05-09T20:26:37.500+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -497> 2024-05-09T20:26:37.500+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6164 payload 274 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -496> 2024-05-09T20:26:37.500+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -495> 2024-05-09T20:26:37.500+0000 7faa591e8640 1 monclient: found mon.noname-c 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -494> 2024-05-09T20:26:37.500+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -493> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -492> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr: -491> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: dump: 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.650 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr: -490> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr: -489> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr: -488> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr: -487> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr: -486> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr: -485> 2024-05-09T20:26:37.500+0000 7fa947e07640 10 monclient: dump: 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.651 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: -484> 2024-05-09T20:26:37.500+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6164 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: -483> 2024-05-09T20:26:37.500+0000 7faa5bed5980 10 monclient: get_monmap_and_config success 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: -482> 2024-05-09T20:26:37.500+0000 7faa5bed5980 4 set_mon_vals no callback set 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: -481> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: shutdown 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: -480> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 asok(0x55c7a50a1430) unregister_commands rotate-key 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: -479> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: build_initial_monmap 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: -478> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 build_initial for_mkfs: 0 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr: -477> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: monmap: 2024-05-09T20:26:37.652 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr: -476> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding auth protocol: cephx 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr: -475> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding auth protocol: cephx 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr: -474> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding auth protocol: cephx 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr: -473> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr: -472> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr: -471> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.653 INFO:tasks.workunit.client.0.smithi063.stderr: -470> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -469> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -468> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -467> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -466> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -465> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -464> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -463> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: crc 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -462> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52b2df0) adding con mode: secure 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -461> 2024-05-09T20:26:37.504+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -460> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: starting msgr at 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -459> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: starting objecter 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -458> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command objecter_requests hook 0x55c7a52b3a90 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -457> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: setting wanted keys 2024-05-09T20:26:37.654 INFO:tasks.workunit.client.0.smithi063.stderr: -456> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: calling monclient init 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -455> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: init 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -454> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding auth protocol: cephx 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -453> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding auth protocol: cephx 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -452> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding auth protocol: cephx 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -451> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -450> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -449> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -448> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -447> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -446> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -445> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -444> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -443> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -442> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -441> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: crc 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -440> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 AuthRegistry(0x55c7a52ad020) adding con mode: secure 2024-05-09T20:26:37.655 INFO:tasks.workunit.client.0.smithi063.stderr: -439> 2024-05-09T20:26:37.504+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -438> 2024-05-09T20:26:37.504+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -437> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command rotate-key hook 0x55c7a52ad168 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -436> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _reopen_session rank -1 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -435> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _add_conns ranks=[0,2,1] 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -434> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-a con 0x55c7a5138fb0 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -433> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-c con 0x55c7a515a0f0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -432> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-b con 0x55c7a51655a0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -431> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -430> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -429> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -428> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): _renew_subs 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -427> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient(hunting): authenticate will time out at 2004.645745s 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -426> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): get_auth_request con 0x55c7a515a0f0 auth_method 0 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -425> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.656 INFO:tasks.workunit.client.0.smithi063.stderr: -424> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -423> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -422> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): get_auth_request con 0x55c7a5138fb0 auth_method 0 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -421> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -420> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -419> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -418> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -417> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -416> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -415> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -414> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -413> 2024-05-09T20:26:37.504+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -412> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6167 payload 824 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -411> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -410> 2024-05-09T20:26:37.504+0000 7faa591e8640 1 monclient: found mon.noname-c 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -409> 2024-05-09T20:26:37.504+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:26:37.657 INFO:tasks.workunit.client.0.smithi063.stderr: -408> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr: -407> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr: -406> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: dump: 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr: -405> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr: -404> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.658 INFO:tasks.workunit.client.0.smithi063.stderr: -403> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr: -402> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr: -401> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr: -400> 2024-05-09T20:26:37.504+0000 7fa946604640 10 monclient: dump: 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr: -399> 2024-05-09T20:26:37.504+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6167 2024-05-09T20:26:37.659 INFO:tasks.workunit.client.0.smithi063.stderr: -398> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -397> 2024-05-09T20:26:37.504+0000 7fa947e07640 4 set_mon_vals no callback set 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -396> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -395> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -394> 2024-05-09T20:26:37.504+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -393> 2024-05-09T20:26:37.504+0000 7faa5bed5980 1 librados: init done 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -392> 2024-05-09T20:26:37.508+0000 7fa946604640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -391> 2024-05-09T20:26:37.508+0000 7fa946604640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -390> 2024-05-09T20:26:37.508+0000 7fa946604640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -389> 2024-05-09T20:26:37.508+0000 7faa599e9640 10 monclient: get_auth_request con 0x7fa940066d60 auth_method 0 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -388> 2024-05-09T20:26:37.508+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x55c7a52d7490 auth_method 0 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -387> 2024-05-09T20:26:37.508+0000 7faa599e9640 10 monclient: get_auth_request con 0x55c7a52f79e0 auth_method 0 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -386> 2024-05-09T20:26:37.508+0000 7faa591e8640 10 monclient: get_auth_request con 0x55c7a52fbbd0 auth_method 0 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -385> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x55c7a52ff8b0 auth_method 0 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -384> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: build_initial_monmap 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -383> 2024-05-09T20:26:37.512+0000 7faa5bed5980 1 build_initial for_mkfs: 0 2024-05-09T20:26:37.660 INFO:tasks.workunit.client.0.smithi063.stderr: -382> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: monmap: 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr: -381> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding auth protocol: cephx 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr: -380> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding auth protocol: cephx 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr: -379> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding auth protocol: cephx 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr: -378> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.661 INFO:tasks.workunit.client.0.smithi063.stderr: -377> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -376> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -375> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -374> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -373> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -372> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -371> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -370> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -369> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -368> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: crc 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -367> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a5303960) adding con mode: secure 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -366> 2024-05-09T20:26:37.512+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -365> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command objecter_requests cmddesc objecter_requests hook 0x55c7a5308ed0 EEXIST 2024-05-09T20:26:37.662 INFO:tasks.workunit.client.0.smithi063.stderr: -364> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: init 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -363> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding auth protocol: cephx 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -362> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding auth protocol: cephx 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -361> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding auth protocol: cephx 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -360> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -359> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -358> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -357> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -356> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -355> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -354> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -353> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -352> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -351> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -350> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: crc 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -349> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53027e8) adding con mode: secure 2024-05-09T20:26:37.663 INFO:tasks.workunit.client.0.smithi063.stderr: -348> 2024-05-09T20:26:37.512+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -347> 2024-05-09T20:26:37.512+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -346> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command rotate-key cmddesc rotate-key hook 0x55c7a5302930 EEXIST 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -345> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _reopen_session rank -1 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -344> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -343> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-a con 0x55c7a532db30 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -342> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-b con 0x55c7a532df10 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -341> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-c con 0x55c7a53290d0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -340> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -339> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -338> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -337> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): _renew_subs 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -336> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient(hunting): authenticate will time out at 2004.655014s 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -335> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): get_auth_request con 0x55c7a532db30 auth_method 0 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -334> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -333> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.664 INFO:tasks.workunit.client.0.smithi063.stderr: -332> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -331> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): get_auth_request con 0x55c7a53290d0 auth_method 0 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -330> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -329> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -328> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -327> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -326> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -325> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -324> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -323> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -322> 2024-05-09T20:26:37.512+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -321> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient(hunting): get_auth_request con 0x55c7a532df10 auth_method 0 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -320> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -319> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -318> 2024-05-09T20:26:37.512+0000 7faa5a1ea640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -317> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6531 payload 824 2024-05-09T20:26:37.665 INFO:tasks.workunit.client.0.smithi063.stderr: -316> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr: -315> 2024-05-09T20:26:37.512+0000 7faa591e8640 1 monclient: found mon.noname-a 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr: -314> 2024-05-09T20:26:37.512+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr: -313> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr: -312> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr: -311> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: dump: 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.666 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -310> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -309> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -308> 2024-05-09T20:26:37.512+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6531 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -307> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -306> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -305> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -304> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -303> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -302> 2024-05-09T20:26:37.512+0000 7faa5a9eb640 4 set_mon_vals no callback set 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr: -301> 2024-05-09T20:26:37.512+0000 7fa93f7fe640 10 monclient: dump: 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.667 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -300> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -299> 2024-05-09T20:26:37.512+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -298> 2024-05-09T20:26:37.516+0000 7fa93f7fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -297> 2024-05-09T20:26:37.516+0000 7fa93f7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -296> 2024-05-09T20:26:37.516+0000 7fa93f7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -295> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x7fa934066ca0 auth_method 0 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -294> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: get_monmap_and_config 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -293> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: build_initial_monmap 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -292> 2024-05-09T20:26:37.516+0000 7faa5bed5980 1 build_initial for_mkfs: 0 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr: -291> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: monmap: 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:26:37.668 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: -290> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding auth protocol: cephx 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: -289> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding auth protocol: cephx 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: -288> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding auth protocol: cephx 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: -287> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: -286> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: -285> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: -284> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.669 INFO:tasks.workunit.client.0.smithi063.stderr: -283> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -282> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -281> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -280> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -279> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -278> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -277> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: crc 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -276> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534b680) adding con mode: secure 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -275> 2024-05-09T20:26:37.516+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -274> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: init 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -273> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding auth protocol: cephx 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -272> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding auth protocol: cephx 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -271> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding auth protocol: cephx 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -270> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.670 INFO:tasks.workunit.client.0.smithi063.stderr: -269> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -268> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -267> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -266> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -265> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -264> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -263> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -262> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -261> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -260> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: crc 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -259> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25b00) adding con mode: secure 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -258> 2024-05-09T20:26:37.516+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -257> 2024-05-09T20:26:37.516+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.671 INFO:tasks.workunit.client.0.smithi063.stderr: -256> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command rotate-key cmddesc rotate-key hook 0x7fffa3d25c48 EEXIST 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -255> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: _reopen_session rank -1 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -254> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: _add_conns ranks=[0,2,1] 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -253> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-a con 0x55c7a537f720 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -252> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-c con 0x55c7a537fb00 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -251> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-b con 0x55c7a5379990 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -250> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -249> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -248> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -247> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): _renew_subs 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -246> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient(hunting): authenticate will time out at 2004.658870s 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -245> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): get_auth_request con 0x55c7a537f720 auth_method 0 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -244> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -243> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -242> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -241> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): get_auth_request con 0x55c7a537fb00 auth_method 0 2024-05-09T20:26:37.672 INFO:tasks.workunit.client.0.smithi063.stderr: -240> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -239> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -238> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -237> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -236> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -235> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -234> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -233> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -232> 2024-05-09T20:26:37.516+0000 7faa5a1ea640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -231> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6534 payload 274 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -230> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -229> 2024-05-09T20:26:37.516+0000 7faa591e8640 1 monclient: found mon.noname-a 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -228> 2024-05-09T20:26:37.516+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -227> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.673 INFO:tasks.workunit.client.0.smithi063.stderr: -226> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr: -225> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: dump: 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr: -224> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.674 INFO:tasks.workunit.client.0.smithi063.stderr: -223> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr: -222> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr: -221> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr: -220> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr: -219> 2024-05-09T20:26:37.516+0000 7fa93d7fa640 10 monclient: dump: 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr: -218> 2024-05-09T20:26:37.516+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6534 2024-05-09T20:26:37.675 INFO:tasks.workunit.client.0.smithi063.stderr: -217> 2024-05-09T20:26:37.516+0000 7faa5bed5980 10 monclient: get_monmap_and_config success 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr: -216> 2024-05-09T20:26:37.516+0000 7faa5bed5980 4 set_mon_vals no callback set 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr: -215> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: shutdown 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr: -214> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: build_initial_monmap 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr: -213> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 build_initial for_mkfs: 0 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr: -212> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: monmap: 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:26:37.676 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -211> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding auth protocol: cephx 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -210> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding auth protocol: cephx 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -209> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding auth protocol: cephx 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -208> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -207> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -206> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -205> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -204> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -203> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -202> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -201> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -200> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.677 INFO:tasks.workunit.client.0.smithi063.stderr: -199> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -198> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: crc 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -197> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a53505d0) adding con mode: secure 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -196> 2024-05-09T20:26:37.520+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -195> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: starting msgr at 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -194> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: starting objecter 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -193> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command objecter_requests cmddesc objecter_requests hook 0x55c7a5328820 EEXIST 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -192> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: setting wanted keys 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -191> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: calling monclient init 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -190> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: init 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -189> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding auth protocol: cephx 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -188> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding auth protocol: cephx 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -187> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding auth protocol: cephx 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -186> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -185> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.678 INFO:tasks.workunit.client.0.smithi063.stderr: -184> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -183> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -182> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -181> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -180> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -179> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -178> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -177> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -176> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: crc 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -175> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 AuthRegistry(0x55c7a534a850) adding con mode: secure 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -174> 2024-05-09T20:26:37.520+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -173> 2024-05-09T20:26:37.520+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -172> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command rotate-key cmddesc rotate-key hook 0x55c7a534a998 EEXIST 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -171> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _reopen_session rank -1 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -170> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -169> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-a con 0x55c7a537f720 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:26:37.679 INFO:tasks.workunit.client.0.smithi063.stderr: -168> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-b con 0x55c7a537d100 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -167> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): picked mon.noname-c con 0x55c7a538c4b0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -166> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -165> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -164> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -163> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): _renew_subs 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -162> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient(hunting): authenticate will time out at 2004.661443s 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -161> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): get_auth_request con 0x55c7a537f720 auth_method 0 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -160> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -159> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -158> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -157> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): get_auth_request con 0x55c7a538c4b0 auth_method 0 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -156> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -155> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:26:37.680 INFO:tasks.workunit.client.0.smithi063.stderr: -154> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -153> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -152> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -151> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -150> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -149> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -148> 2024-05-09T20:26:37.520+0000 7faa599e9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -147> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient(hunting): handle_auth_done global_id 6537 payload 824 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -146> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient: _finish_hunting 0 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -145> 2024-05-09T20:26:37.520+0000 7faa591e8640 1 monclient: found mon.noname-a 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -144> 2024-05-09T20:26:37.520+0000 7faa591e8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -143> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -142> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr: -141> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: dump: 2024-05-09T20:26:37.681 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr: -140> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: _finish_auth 0 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr: -139> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: _check_auth_tickets 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr: -138> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: handle_config config(0 keys) 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr: -137> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:26:37.682 INFO:tasks.workunit.client.0.smithi063.stderr: -136> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr: -135> 2024-05-09T20:26:37.520+0000 7fa92b7fe640 10 monclient: dump: 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr: -134> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 monclient: authenticate success, global_id 6537 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr: -133> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr: -132> 2024-05-09T20:26:37.520+0000 7fa93d7fa640 4 set_mon_vals no callback set 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr: -131> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.683 INFO:tasks.workunit.client.0.smithi063.stderr: -130> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _renew_subs 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -129> 2024-05-09T20:26:37.520+0000 7faa5bed5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -128> 2024-05-09T20:26:37.520+0000 7faa5bed5980 1 librados: init done 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -127> 2024-05-09T20:26:37.520+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cr dump hook 0x55c7a5384608 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -126> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -125> 2024-05-09T20:26:37.524+0000 7fa92b7fe640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -124> 2024-05-09T20:26:37.524+0000 7fa92b7fe640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -123> 2024-05-09T20:26:37.524+0000 7fa92b7fe640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -122> 2024-05-09T20:26:37.524+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x7fa924066d90 auth_method 0 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -121> 2024-05-09T20:26:37.524+0000 7faa599e9640 10 monclient: get_auth_request con 0x55c7a5398f70 auth_method 0 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -120> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -119> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 realm 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -118> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -117> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -116> 2024-05-09T20:26:37.524+0000 7faa5bed5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -115> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.684 INFO:tasks.workunit.client.0.smithi063.stderr: -114> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -113> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -112> 2024-05-09T20:26:37.524+0000 7faa591e8640 10 monclient: get_auth_request con 0x55c7a539ce30 auth_method 0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -111> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -110> 2024-05-09T20:26:37.524+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -109> 2024-05-09T20:26:37.524+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x55c7a53a06b0 auth_method 0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -108> 2024-05-09T20:26:37.528+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -107> 2024-05-09T20:26:37.528+0000 7faa5bed5980 20 searching for the correct realm 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -106> 2024-05-09T20:26:37.528+0000 7faa599e9640 10 monclient: get_auth_request con 0x55c7a53a5490 auth_method 0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -105> 2024-05-09T20:26:37.528+0000 7faa591e8640 10 monclient: get_auth_request con 0x7faa4c0313f0 auth_method 0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -104> 2024-05-09T20:26:37.528+0000 7faa5a1ea640 10 monclient: get_auth_request con 0x7faa500c8020 auth_method 0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -103> 2024-05-09T20:26:37.532+0000 7faa599e9640 10 monclient: get_auth_request con 0x7faa54038920 auth_method 0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -102> 2024-05-09T20:26:37.532+0000 7faa599e9640 0 -- 172.21.15.63:0/2733727874 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7faa54038920 msgr2=0x7faa5400f430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -101> 2024-05-09T20:26:37.532+0000 7fa92b7fe640 1 client.6537.objecter ms_handle_reset 0x7faa54038920 session 0x7faa54049040 osd.4 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -100> 2024-05-09T20:26:37.532+0000 7fa92b7fe640 10 monclient: _renew_subs 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -99> 2024-05-09T20:26:37.532+0000 7fa92b7fe640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:26:37.685 INFO:tasks.workunit.client.0.smithi063.stderr: -98> 2024-05-09T20:26:37.532+0000 7faa599e9640 10 monclient: get_auth_request con 0x7fa9240774c0 auth_method 0 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -97> 2024-05-09T20:26:37.536+0000 7faa591e8640 10 monclient: get_auth_request con 0x7faa4c02f3e0 auth_method 0 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -96> 2024-05-09T20:26:37.540+0000 7faa599e9640 0 -- 172.21.15.63:0/2733727874 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fa9240774c0 msgr2=0x7fa9240798f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -95> 2024-05-09T20:26:37.540+0000 7fa92b7fe640 1 client.6537.objecter ms_handle_reset 0x7fa9240774c0 session 0x7faa54049040 osd.4 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -94> 2024-05-09T20:26:37.540+0000 7faa599e9640 10 monclient: get_auth_request con 0x7faa54038920 auth_method 0 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -93> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -92> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -91> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -90> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -89> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -88> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -87> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -86> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -85> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -84> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.686 INFO:tasks.workunit.client.0.smithi063.stderr: -83> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -82> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -81> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 zone default found 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -80> 2024-05-09T20:26:37.548+0000 7faa5bed5980 4 Realm: () 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -79> 2024-05-09T20:26:37.548+0000 7faa5bed5980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -78> 2024-05-09T20:26:37.548+0000 7faa5bed5980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -77> 2024-05-09T20:26:37.548+0000 7faa5bed5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -76> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 zonegroup default 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -75> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -74> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -73> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -72> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -71> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -70> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -69> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 started sync module instance, tier type = 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -68> 2024-05-09T20:26:37.548+0000 7faa5bed5980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:37.687 INFO:tasks.workunit.client.0.smithi063.stderr: -67> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=0 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -66> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=1 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -65> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=2 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -64> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=3 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -63> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=4 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -62> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=5 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -61> 2024-05-09T20:26:37.564+0000 7faa5bed5980 20 add_watcher() i=6 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -60> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 add_watcher() i=7 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -59> 2024-05-09T20:26:37.568+0000 7faa5bed5980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -58> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding auth protocol: cephx 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -57> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding auth protocol: cephx 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -56> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding auth protocol: cephx 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -55> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -54> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -53> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -52> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.688 INFO:tasks.workunit.client.0.smithi063.stderr: -51> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -50> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -49> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -48> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -47> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -46> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -45> 2024-05-09T20:26:37.568+0000 7fa90b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -44> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: crc 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -43> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 AuthRegistry(0x7fffa3d25f90) adding con mode: secure 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -42> 2024-05-09T20:26:37.568+0000 7faa5bed5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -41> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -40> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -39> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cache list hook 0x55c7a5351a80 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -38> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cache inspect hook 0x55c7a5351a80 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -37> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cache erase hook 0x55c7a5351a80 2024-05-09T20:26:37.689 INFO:tasks.workunit.client.0.smithi063.stderr: -36> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command cache zap hook 0x55c7a5351a80 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -35> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 note: GC not initialized 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -34> 2024-05-09T20:26:37.568+0000 7fa90a7fc640 20 reqs_thread_entry: start 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -33> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command sync trace show hook 0x7faa500043b0 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -32> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command sync trace history hook 0x7faa500043b0 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -31> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command sync trace active hook 0x7faa500043b0 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -30> 2024-05-09T20:26:37.568+0000 7faa5bed5980 5 asok(0x55c7a50a1430) register_command sync trace active_short hook 0x7faa500043b0 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -29> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -28> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 Filter name: none 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -27> 2024-05-09T20:26:37.568+0000 7fa87bfff640 20 reqs_thread_entry: start 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -26> 2024-05-09T20:26:37.568+0000 7faa5bed5980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -25> 2024-05-09T20:26:37.568+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -24> 2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -23> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -22> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -21> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 : miss 2024-05-09T20:26:37.690 INFO:tasks.workunit.client.0.smithi063.stderr: -20> 2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -19> 2024-05-09T20:26:37.572+0000 7faa5bed5980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -18> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -17> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -16> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -15> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -14> 2024-05-09T20:26:37.572+0000 7faa5bed5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -13> 2024-05-09T20:26:37.580+0000 7faa5bed5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -12> 2024-05-09T20:26:37.580+0000 7faa5bed5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -11> 2024-05-09T20:26:37.580+0000 7faa5bed5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -10> 2024-05-09T20:26:37.580+0000 7faa5bed5980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23, ofs0, ns] 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -9> 2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431631 cookie 94315958939888 notifier 6537 bl.length()=907 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -8> 2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -7> 2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:37.691 INFO:tasks.workunit.client.0.smithi063.stderr: -6> 2024-05-09T20:26:37.584+0000 7fa93cff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: -5> 2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_sources= new_sources= 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: -4> 2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential sources added=[] removed=[] 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: -3> 2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_dests= new_dests= 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: -2> 2024-05-09T20:26:37.584+0000 7faa5bed5980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential dests added=[] removed=[] 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: -1> 2024-05-09T20:26:37.584+0000 7faa5bed5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7faa5bed5980 time 2024-05-09T20:26:37.588794+0000 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7faa5f7abe25] 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7faa5f7abfd9] 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c7a2c8dca1] 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ad) [0x55c7a2c9e3ad] 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:26:37.692 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7faa5e615d90] 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 0> 2024-05-09T20:26:37.588+0000 7faa5bed5980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7faa5bed5980 thread_name:radosgw-admin 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7faa5e62e520] 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7faa5f7abe7f] 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7faa5f7abfd9] 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55c7a2c8dca1] 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x2ad) [0x55c7a2c9e3ad] 2024-05-09T20:26:37.693 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7faa5e615d90] 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr:--- logging levels --- 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 none 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 lockdep 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 context 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 crush 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_balancer 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_locker 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log 2024-05-09T20:26:37.694 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log_expire 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_migrator 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 3/ 5 mds_quiesce 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 buffer 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 timer 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 filer 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 striper 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 objecter 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rados 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_mirror 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_replay 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_pwl 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 journaler 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objectcacher 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 immutable_obj_cache 2024-05-09T20:26:37.695 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 client 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 osd 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 optracker 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objclass 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 filestore 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 journal 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 ms 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mon 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 0/10 monc 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 paxos 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 tp 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 auth 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 crypto 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 finisher 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 reserver 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 heartbeatmap 2024-05-09T20:26:37.696 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 perfcounter 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 20/20 rgw 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_sync 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_datacache 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_access 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_dbstore 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_flight 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_lifecycle 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_notification 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 javaclient 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 asok 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 throttle 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 refs 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 compressor 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluestore 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluefs 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 bdev 2024-05-09T20:26:37.697 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 kstore 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 4/ 5 rocksdb 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 fuse 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 2/ 5 mgr 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mgrc 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 dpdk 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 eventtrace 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 prioritycache 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 test 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephfs_mirror 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephsqlite 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 crimson_interrupt 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_onode 2024-05-09T20:26:37.698 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_odata 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_omap 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_tm 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_t 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cleaner 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_epm 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_lba 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cache 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_journal 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_device 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_backref 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 alienstore 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mclock 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cyanstore 2024-05-09T20:26:37.699 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 ceph_exporter 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 memstore 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 trace 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 ceph_dedup 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: -2/-2 (syslog threshold) 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 99/99 (stderr threshold) 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa87bfff640 / http_manager 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa90a7fc640 / http_manager 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa90b7fe640 / rgw_dt_lg_renew 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa92b7fe640 / ms_dispatch 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa93cff9640 / io_context_pool 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa93d7fa640 / io_context_pool 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa93f7fe640 / ms_dispatch 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa946604640 / ms_dispatch 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7fa947e07640 / io_context_pool 2024-05-09T20:26:37.700 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa591e8640 / msgr-worker-2 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa599e9640 / msgr-worker-1 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa5a1ea640 / msgr-worker-0 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa5a9eb640 / io_context_pool 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr: 7faa5bed5980 / radosgw-admin 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr: max_recent 500 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr: max_new 1000 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr: log_file /var/lib/ceph/crash/2024-05-09T20:26:37.590752Z_d61ff593-12ca-4c0c-9a2c-f94547a252fc/log 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr:--- end dump of recent events --- 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr:Aborted 2024-05-09T20:26:37.701 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 20 realm 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.664+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.668+0000 7f6827fcf980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.668+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.668+0000 7f6827fcf980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:37.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.668+0000 7f6827fcf980 20 searching for the correct realm 2024-05-09T20:26:37.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.672+0000 7f6825ae3640 0 -- 172.21.15.63:0/3409281703 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f68200d0050 msgr2=0x7f68200d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:37.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.888+0000 7f6825ae3640 0 -- 172.21.15.63:0/3409281703 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f68200d0050 msgr2=0x7f68200d1430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:37.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:37.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:37.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:37.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:37.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:37.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:37.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:37.893 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.900+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:37.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 zone default found 2024-05-09T20:26:37.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 4 Realm: () 2024-05-09T20:26:37.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:37.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:37.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:37.894 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 zonegroup default 2024-05-09T20:26:37.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.895 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:37.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 started sync module instance, tier type = 2024-05-09T20:26:37.896 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.904+0000 7f6827fcf980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:37.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 20 add_watcher() i=0 2024-05-09T20:26:37.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 20 add_watcher() i=1 2024-05-09T20:26:37.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 20 add_watcher() i=2 2024-05-09T20:26:37.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 20 add_watcher() i=3 2024-05-09T20:26:37.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 20 add_watcher() i=4 2024-05-09T20:26:37.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 20 add_watcher() i=5 2024-05-09T20:26:37.918 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 20 add_watcher() i=6 2024-05-09T20:26:37.919 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 20 add_watcher() i=7 2024-05-09T20:26:37.919 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.928+0000 7f6827fcf980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:37.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f66d77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:37.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6827fcf980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:37.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6827fcf980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:37.923 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6827fcf980 5 note: GC not initialized 2024-05-09T20:26:37.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f66d67fc640 20 reqs_thread_entry: start 2024-05-09T20:26:37.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6827fcf980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:37.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6827fcf980 20 Filter name: none 2024-05-09T20:26:37.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6647fff640 20 reqs_thread_entry: start 2024-05-09T20:26:37.924 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6827fcf980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:37.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.932+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.936+0000 7f6827fcf980 20 rados_obj.operate() r=0 bl.length=435 2024-05-09T20:26:37.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.936+0000 7f6827fcf980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:37.925 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.936+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.936+0000 7f6827fcf980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:37.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.936+0000 7f6827fcf980 20 rados->read ofs=0 len=0 2024-05-09T20:26:37.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.936+0000 7f6827fcf980 20 rados_obj.operate() r=0 bl.length=435 2024-05-09T20:26:37.926 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.936+0000 7f6827fcf980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:26:37.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.940+0000 7f66d77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:37.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.944+0000 7f6827fcf980 20 remove_watcher() i=0 2024-05-09T20:26:37.934 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.944+0000 7f6827fcf980 2 removed watcher, disabling cache 2024-05-09T20:26:37.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.944+0000 7f6827fcf980 20 remove_watcher() i=1 2024-05-09T20:26:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.948+0000 7f6827fcf980 20 remove_watcher() i=2 2024-05-09T20:26:37.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.952+0000 7f6827fcf980 20 remove_watcher() i=3 2024-05-09T20:26:37.944 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.952+0000 7f6827fcf980 20 remove_watcher() i=4 2024-05-09T20:26:37.945 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.956+0000 7f6827fcf980 20 remove_watcher() i=5 2024-05-09T20:26:37.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.956+0000 7f6827fcf980 20 remove_watcher() i=6 2024-05-09T20:26:37.948 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:37.956+0000 7f6827fcf980 20 remove_watcher() i=7 2024-05-09T20:26:37.965 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:37.965 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:26:37.965 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-block-writes id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:26:37.965 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:26:38.075 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:26:38.112 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:26:38.112 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:26:38.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:38.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 20 realm 2024-05-09T20:26:38.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:38.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:26:38.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:38.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.168+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.172+0000 7f7c8fe65980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:38.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.172+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.172+0000 7f7c8fe65980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:26:38.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.172+0000 7f7c8fe65980 20 searching for the correct realm 2024-05-09T20:26:38.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.176+0000 7f7c8d979640 0 -- 172.21.15.63:0/2535318482 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f7c8003b040 msgr2=0x7f7c8006d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:38.384 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.392+0000 7f7c8d979640 0 -- 172.21.15.63:0/2535318482 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f7c8003b040 msgr2=0x7f7c8006d5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:26:38.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:26:38.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:26:38.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:26:38.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:26:38.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:26:38.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:26:38.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:38.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.404+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:26:38.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:26:38.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 zone default found 2024-05-09T20:26:38.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 4 Realm: () 2024-05-09T20:26:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:26:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:26:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:26:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 zonegroup default 2024-05-09T20:26:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.400 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:38.400 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:26:38.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 started sync module instance, tier type = 2024-05-09T20:26:38.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.408+0000 7f7c8fe65980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:26:38.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.432+0000 7f7c8fe65980 20 add_watcher() i=0 2024-05-09T20:26:38.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.432+0000 7f7c8fe65980 20 add_watcher() i=1 2024-05-09T20:26:38.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.432+0000 7f7c8fe65980 20 add_watcher() i=2 2024-05-09T20:26:38.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.432+0000 7f7c8fe65980 20 add_watcher() i=3 2024-05-09T20:26:38.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.432+0000 7f7c8fe65980 20 add_watcher() i=4 2024-05-09T20:26:38.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.432+0000 7f7c8fe65980 20 add_watcher() i=5 2024-05-09T20:26:38.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.432+0000 7f7c8fe65980 20 add_watcher() i=6 2024-05-09T20:26:38.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.436+0000 7f7c8fe65980 20 add_watcher() i=7 2024-05-09T20:26:38.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.436+0000 7f7c8fe65980 2 all 8 watchers are set, enabling cache 2024-05-09T20:26:38.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7b3f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:38.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:26:38.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:26:38.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 5 note: GC not initialized 2024-05-09T20:26:38.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7b3e7fc640 20 reqs_thread_entry: start 2024-05-09T20:26:38.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 20 init_complete bucket index max shards: 11 2024-05-09T20:26:38.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 20 Filter name: none 2024-05-09T20:26:38.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7ab3fff640 20 reqs_thread_entry: start 2024-05-09T20:26:38.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-09T20:26:38.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:26:38.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-09T20:26:38.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-09T20:26:38.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 : miss 2024-05-09T20:26:38.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.440+0000 7f7c8fe65980 20 rados->read ofs=0 len=0 2024-05-09T20:26:38.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.444+0000 7f7c8fe65980 20 rados_obj.operate() r=0 bl.length=435 2024-05-09T20:26:38.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.444+0000 7f7c8fe65980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:26:38.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.444+0000 7f7c8fe65980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:26:38.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.444+0000 7f7c8fe65980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:26:38.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.444+0000 7f7c8fe65980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:26:38.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.444+0000 7f7c8fe65980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 2024-05-09T20:26:38.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.444+0000 7f7c8fe65980 0 INFO: RGWReshardLock::lock found lock on abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to be held by another RGW process; skipping for now 2024-05-09T20:26:38.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.444+0000 7f7b3f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:26:38.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.452+0000 7f7c8fe65980 20 remove_watcher() i=0 2024-05-09T20:26:38.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.452+0000 7f7c8fe65980 2 removed watcher, disabling cache 2024-05-09T20:26:38.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.452+0000 7f7c8fe65980 20 remove_watcher() i=1 2024-05-09T20:26:38.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.456+0000 7f7c8fe65980 20 remove_watcher() i=2 2024-05-09T20:26:38.449 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.456+0000 7f7c8fe65980 20 remove_watcher() i=3 2024-05-09T20:26:38.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.460+0000 7f7c8fe65980 20 remove_watcher() i=4 2024-05-09T20:26:38.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.460+0000 7f7c8fe65980 20 remove_watcher() i=5 2024-05-09T20:26:38.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.464+0000 7f7c8fe65980 20 remove_watcher() i=6 2024-05-09T20:26:38.455 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:26:38.464+0000 7f7c8fe65980 20 remove_watcher() i=7 2024-05-09T20:26:38.472 INFO:tasks.workunit.client.0.smithi063.stderr:waiting 30 seconds for reshard lock to expire... 2024-05-09T20:27:08.498 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-block-writes --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:27:08.536 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:08.536 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:08.583 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 20 realm 2024-05-09T20:27:08.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:08.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.592+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.596+0000 7f102ae44980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:08.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.596+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.596+0000 7f102ae44980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:08.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.596+0000 7f102ae44980 20 searching for the correct realm 2024-05-09T20:27:08.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.600+0000 7f1029159640 0 -- 172.21.15.63:0/4014783198 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f1014086c20 msgr2=0x7f101405d6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:08.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.608+0000 7f1028157640 0 -- 172.21.15.63:0/4014783198 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f10180d5920 msgr2=0x7f10180aa3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:08.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 zone default found 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 4 Realm: () 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 zonegroup default 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.616+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.620+0000 7f102ae44980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.620+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.620+0000 7f102ae44980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.620+0000 7f102ae44980 20 started sync module instance, tier type = 2024-05-09T20:27:08.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.620+0000 7f102ae44980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:08.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 20 add_watcher() i=0 2024-05-09T20:27:08.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 20 add_watcher() i=1 2024-05-09T20:27:08.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 20 add_watcher() i=2 2024-05-09T20:27:08.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 20 add_watcher() i=3 2024-05-09T20:27:08.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 20 add_watcher() i=4 2024-05-09T20:27:08.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 20 add_watcher() i=5 2024-05-09T20:27:08.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 20 add_watcher() i=6 2024-05-09T20:27:08.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 20 add_watcher() i=7 2024-05-09T20:27:08.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.636+0000 7f102ae44980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:08.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f0edefdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 5 note: GC not initialized 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f0eddfdb640 20 reqs_thread_entry: start 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 20 Filter name: none 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f0e4f7fe640 20 reqs_thread_entry: start 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 cache get: name=default.rgw.meta+root+abort-at-block-writes : miss 2024-05-09T20:27:08.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:27:08.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 cache put: name=default.rgw.meta+root+abort-at-block-writes info.flags=0x11 2024-05-09T20:27:08.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 adding default.rgw.meta+root+abort-at-block-writes to cache LRU end 2024-05-09T20:27:08.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 : miss 2024-05-09T20:27:08.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 20 rados_obj.operate() r=0 bl.length=435 2024-05-09T20:27:08.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:27:08.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:27:08.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:08.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:08.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.640+0000 7f102ae44980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 2024-05-09T20:27:08.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.644+0000 7f102ae44980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-05-09T20:27:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.656+0000 7f102ae44980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:27:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.656+0000 7f102ae44980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:27:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.656+0000 7f102ae44980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:08.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.656+0000 7f102ae44980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23, ofs0, ns] 2024-05-09T20:27:08.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.656+0000 7f0eff7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431632 cookie 93933040337888 notifier 6591 bl.length()=907 2024-05-09T20:27:08.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.656+0000 7f0eff7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:27:08.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.656+0000 7f0eff7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:27:08.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.656+0000 7f0eff7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:08.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.660+0000 7f102ae44980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_sources= new_sources= 2024-05-09T20:27:08.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.660+0000 7f102ae44980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential sources added=[] removed=[] 2024-05-09T20:27:08.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.660+0000 7f102ae44980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_dests= new_dests= 2024-05-09T20:27:08.650 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.660+0000 7f102ae44980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential dests added=[] removed=[] 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.0 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.0 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.1 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.1 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.2 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.2 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.3 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.3 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.4 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.4 2024-05-09T20:27:08.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.5 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.5 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.6 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.6 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.7 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.7 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.8 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.8 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.9 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.9 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.10 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.10 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.11 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2.11 2024-05-09T20:27:08.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.0 2024-05-09T20:27:08.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.664+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.1 2024-05-09T20:27:08.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.668+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.2 2024-05-09T20:27:08.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.668+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.3 2024-05-09T20:27:08.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.668+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.4 2024-05-09T20:27:08.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.668+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.5 2024-05-09T20:27:08.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.668+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.6 2024-05-09T20:27:08.662 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.672+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.7 2024-05-09T20:27:08.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.672+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.8 2024-05-09T20:27:08.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.672+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.9 2024-05-09T20:27:08.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.676+0000 7f102ae44980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23.10 2024-05-09T20:27:08.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.680+0000 7f102ae44980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 : hit (requested=0x17, cached=0x17) 2024-05-09T20:27:08.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.680+0000 7f102ae44980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 2024-05-09T20:27:08.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.684+0000 7f102ae44980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:27:08.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.684+0000 7f102ae44980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:27:08.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.684+0000 7f102ae44980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:08.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.684+0000 7f102ae44980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23, ofs0, ns] 2024-05-09T20:27:08.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.684+0000 7f0efeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431633 cookie 93933040337888 notifier 6591 bl.length()=875 2024-05-09T20:27:08.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.684+0000 7f0efeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 info.flags=0x17 2024-05-09T20:27:08.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.684+0000 7f0efeffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-block-writes:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 to cache LRU end 2024-05-09T20:27:08.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.684+0000 7f0efeffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:08.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.688+0000 7f102ae44980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_sources= new_sources= 2024-05-09T20:27:08.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.688+0000 7f102ae44980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential sources added=[] removed=[] 2024-05-09T20:27:08.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.688+0000 7f102ae44980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): orig_dests= new_dests= 2024-05-09T20:27:08.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.688+0000 7f102ae44980 20 handle_bi_update(): bucket=:abort-at-block-writes[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23]): potential dests added=[] removed=[] 2024-05-09T20:27:08.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.692+0000 7f102ae44980 1 execute INFO: reshard of bucket "abort-at-block-writes" completed successfully 2024-05-09T20:27:08.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.696+0000 7f0edefdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:08.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.700+0000 7f102ae44980 20 remove_watcher() i=0 2024-05-09T20:27:08.693 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.700+0000 7f102ae44980 2 removed watcher, disabling cache 2024-05-09T20:27:08.695 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.704+0000 7f102ae44980 20 remove_watcher() i=1 2024-05-09T20:27:08.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.708+0000 7f102ae44980 20 remove_watcher() i=2 2024-05-09T20:27:08.699 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.708+0000 7f102ae44980 20 remove_watcher() i=3 2024-05-09T20:27:08.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.712+0000 7f102ae44980 20 remove_watcher() i=4 2024-05-09T20:27:08.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.712+0000 7f102ae44980 20 remove_watcher() i=5 2024-05-09T20:27:08.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.712+0000 7f102ae44980 20 remove_watcher() i=6 2024-05-09T20:27:08.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.716+0000 7f102ae44980 20 remove_watcher() i=7 2024-05-09T20:27:08.723 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-block-writes 2024-05-09T20:27:08.751 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:08.751 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:08.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 20 realm 2024-05-09T20:27:08.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:08.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.804+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.808+0000 7fed595e0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:08.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.808+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.808+0000 7fed595e0980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:08.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.808+0000 7fed595e0980 20 searching for the correct realm 2024-05-09T20:27:08.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.812+0000 7fed578f5640 0 -- 172.21.15.63:0/4214845043 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fed4c008050 msgr2=0x7fed4c055ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:08.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.820+0000 7fed578f5640 0 -- 172.21.15.63:0/4214845043 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fec200774c0 msgr2=0x7fec200798f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.828+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:08.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 zone default found 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 4 Realm: () 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 zonegroup default 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:08.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 started sync module instance, tier type = 2024-05-09T20:27:08.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.832+0000 7fed595e0980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:08.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 20 add_watcher() i=0 2024-05-09T20:27:08.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 20 add_watcher() i=1 2024-05-09T20:27:08.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 20 add_watcher() i=2 2024-05-09T20:27:08.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 20 add_watcher() i=3 2024-05-09T20:27:08.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 20 add_watcher() i=4 2024-05-09T20:27:08.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 20 add_watcher() i=5 2024-05-09T20:27:08.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 20 add_watcher() i=6 2024-05-09T20:27:08.839 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 20 add_watcher() i=7 2024-05-09T20:27:08.839 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.848+0000 7fed595e0980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:08.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fec0dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:08.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:08.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:08.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 5 note: GC not initialized 2024-05-09T20:27:08.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fec0cfe1640 20 reqs_thread_entry: start 2024-05-09T20:27:08.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:08.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 Filter name: none 2024-05-09T20:27:08.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7feb867fc640 20 reqs_thread_entry: start 2024-05-09T20:27:08.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:27:08.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:27:08.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:08.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rados_obj.operate() r=0 bl.length=189 2024-05-09T20:27:08.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:08.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 rados_obj.operate() r=0 bl.length=403 2024-05-09T20:27:08.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.852+0000 7fed595e0980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:08.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.856+0000 7fec0dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:08.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.860+0000 7fed595e0980 20 remove_watcher() i=0 2024-05-09T20:27:08.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.860+0000 7fed595e0980 2 removed watcher, disabling cache 2024-05-09T20:27:08.855 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.864+0000 7fed595e0980 20 remove_watcher() i=1 2024-05-09T20:27:08.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.864+0000 7fed595e0980 20 remove_watcher() i=2 2024-05-09T20:27:08.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.868+0000 7fed595e0980 20 remove_watcher() i=3 2024-05-09T20:27:08.860 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.868+0000 7fed595e0980 20 remove_watcher() i=4 2024-05-09T20:27:08.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.872+0000 7fed595e0980 20 remove_watcher() i=5 2024-05-09T20:27:08.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.872+0000 7fed595e0980 20 remove_watcher() i=6 2024-05-09T20:27:08.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:08.872+0000 7fed595e0980 20 remove_watcher() i=7 2024-05-09T20:27:08.878 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:08.878 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:08.878 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-block-writes id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.23 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-09T20:27:08.878 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:09.782 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with EIO injected at commit_target_layout 2024-05-09T20:27:09.782 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:11.882 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-09T20:27:11.915 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:11.916 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:11.961 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.968+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:11.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.972+0000 7ff23bbb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:11.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.972+0000 7ff23bbb2980 20 realm 2024-05-09T20:27:11.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.972+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:11.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.972+0000 7ff23bbb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:11.963 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.972+0000 7ff23bbb2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:11.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.972+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:11.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.972+0000 7ff23bbb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:11.964 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.972+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:11.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.976+0000 7ff23bbb2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:11.965 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.976+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:11.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.976+0000 7ff23bbb2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:11.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.976+0000 7ff23bbb2980 20 searching for the correct realm 2024-05-09T20:27:11.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:11.976+0000 7ff239ec7640 0 -- 172.21.15.63:0/1061077672 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7ff22c02e060 msgr2=0x7ff22c05a5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:12.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.188+0000 7ff238ec5640 0 -- 172.21.15.63:0/1061077672 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7ff2241019d0 msgr2=0x7ff2240fe7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:12.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.204+0000 7ff23bbb2980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:12.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.204+0000 7ff23bbb2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:12.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.204+0000 7ff23bbb2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:12.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.204+0000 7ff23bbb2980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:12.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.204+0000 7ff23bbb2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:12.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.204+0000 7ff23bbb2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:12.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.204+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.197 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.198 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:12.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:12.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 zone default found 2024-05-09T20:27:12.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 4 Realm: () 2024-05-09T20:27:12.199 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:12.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:12.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:12.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 zonegroup default 2024-05-09T20:27:12.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.200 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.208+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.212+0000 7ff23bbb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.201 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.212+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.212+0000 7ff23bbb2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.212+0000 7ff23bbb2980 20 started sync module instance, tier type = 2024-05-09T20:27:12.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.212+0000 7ff23bbb2980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:12.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 20 add_watcher() i=0 2024-05-09T20:27:12.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 20 add_watcher() i=1 2024-05-09T20:27:12.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 20 add_watcher() i=2 2024-05-09T20:27:12.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 20 add_watcher() i=3 2024-05-09T20:27:12.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 20 add_watcher() i=4 2024-05-09T20:27:12.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 20 add_watcher() i=5 2024-05-09T20:27:12.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 20 add_watcher() i=6 2024-05-09T20:27:12.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 20 add_watcher() i=7 2024-05-09T20:27:12.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.236+0000 7ff23bbb2980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:12.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.240+0000 7ff0ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:12.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.240+0000 7ff23bbb2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:12.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.240+0000 7ff23bbb2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:12.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 5 note: GC not initialized 2024-05-09T20:27:12.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff0ee7e4640 20 reqs_thread_entry: start 2024-05-09T20:27:12.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:12.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 20 Filter name: none 2024-05-09T20:27:12.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff05ffff640 20 reqs_thread_entry: start 2024-05-09T20:27:12.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:12.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:12.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:12.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.244+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.248+0000 7ff23bbb2980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:12.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.248+0000 7ff23bbb2980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.248+0000 7ff23bbb2980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:12.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.248+0000 7ff23bbb2980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:12.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.252+0000 7ff0ef7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:12.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.256+0000 7ff23bbb2980 20 remove_watcher() i=0 2024-05-09T20:27:12.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.256+0000 7ff23bbb2980 2 removed watcher, disabling cache 2024-05-09T20:27:12.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.260+0000 7ff23bbb2980 20 remove_watcher() i=1 2024-05-09T20:27:12.252 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.260+0000 7ff23bbb2980 20 remove_watcher() i=2 2024-05-09T20:27:12.254 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.264+0000 7ff23bbb2980 20 remove_watcher() i=3 2024-05-09T20:27:12.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.264+0000 7ff23bbb2980 20 remove_watcher() i=4 2024-05-09T20:27:12.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.268+0000 7ff23bbb2980 20 remove_watcher() i=5 2024-05-09T20:27:12.259 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.268+0000 7ff23bbb2980 20 remove_watcher() i=6 2024-05-09T20:27:12.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.268+0000 7ff23bbb2980 20 remove_watcher() i=7 2024-05-09T20:27:12.278 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:12.278 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:12.278 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-commit-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:27:12.278 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:12.278 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout 2024-05-09T20:27:12.311 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:12.311 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:12.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.356 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 20 realm 2024-05-09T20:27:12.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:12.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.357 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.364+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.368+0000 7f01b4831980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:12.358 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.368+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.368+0000 7f01b4831980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:12.359 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.368+0000 7f01b4831980 20 searching for the correct realm 2024-05-09T20:27:12.363 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.372+0000 7f01b2345640 0 -- 172.21.15.63:0/4273504768 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f01ac018070 msgr2=0x7f01ac05be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:12.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.588+0000 7f01b2345640 0 -- 172.21.15.63:0/4273504768 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f01ac018070 msgr2=0x7f01ac05be50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:12.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:12.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:12.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:12.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:12.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:12.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:12.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:12.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.600+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:12.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 zone default found 2024-05-09T20:27:12.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 4 Realm: () 2024-05-09T20:27:12.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:12.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:12.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:12.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 zonegroup default 2024-05-09T20:27:12.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 started sync module instance, tier type = 2024-05-09T20:27:12.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.604+0000 7f01b4831980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:12.619 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.628+0000 7f01b4831980 20 add_watcher() i=0 2024-05-09T20:27:12.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.628+0000 7f01b4831980 20 add_watcher() i=1 2024-05-09T20:27:12.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.628+0000 7f01b4831980 20 add_watcher() i=2 2024-05-09T20:27:12.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.628+0000 7f01b4831980 20 add_watcher() i=3 2024-05-09T20:27:12.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.628+0000 7f01b4831980 20 add_watcher() i=4 2024-05-09T20:27:12.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.628+0000 7f01b4831980 20 add_watcher() i=5 2024-05-09T20:27:12.620 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.628+0000 7f01b4831980 20 add_watcher() i=6 2024-05-09T20:27:12.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.632+0000 7f01b4831980 20 add_watcher() i=7 2024-05-09T20:27:12.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.632+0000 7f01b4831980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:12.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7f0068fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:12.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7f01b4831980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:12.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7f01b4831980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:12.627 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7f01b4831980 5 note: GC not initialized 2024-05-09T20:27:12.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7effe37fe640 20 reqs_thread_entry: start 2024-05-09T20:27:12.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7f01b4831980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:12.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7f01b4831980 20 Filter name: none 2024-05-09T20:27:12.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7effe17fa640 20 reqs_thread_entry: start 2024-05-09T20:27:12.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7f01b4831980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-05-09T20:27:12.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.636+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:12.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-05-09T20:27:12.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-05-09T20:27:12.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 : miss 2024-05-09T20:27:12.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:12.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 info.flags=0x17 2024-05-09T20:27:12.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 to cache LRU end 2024-05-09T20:27:12.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:12.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:12.631 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.640+0000 7f01b4831980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 2024-05-09T20:27:12.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.652+0000 7f01b4831980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 info.flags=0x17 2024-05-09T20:27:12.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.652+0000 7f01b4831980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 to cache LRU end 2024-05-09T20:27:12.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.652+0000 7f01b4831980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:12.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.652+0000 7f01b4831980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24, ofs0, ns] 2024-05-09T20:27:12.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.652+0000 7f0095ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431589 cookie 94888268077920 notifier 6651 bl.length()=923 2024-05-09T20:27:12.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.652+0000 7f0095ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 info.flags=0x17 2024-05-09T20:27:12.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.652+0000 7f0095ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 to cache LRU end 2024-05-09T20:27:12.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.652+0000 7f0095ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:12.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.656+0000 7f01b4831980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): orig_sources= new_sources= 2024-05-09T20:27:12.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.656+0000 7f01b4831980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): potential sources added=[] removed=[] 2024-05-09T20:27:12.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.656+0000 7f01b4831980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): orig_dests= new_dests= 2024-05-09T20:27:12.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.656+0000 7f01b4831980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): potential dests added=[] removed=[] 2024-05-09T20:27:12.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.0 2024-05-09T20:27:12.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.0 2024-05-09T20:27:12.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.1 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.1 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.2 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.2 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.3 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.3 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.4 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.4 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.5 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.5 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.6 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.6 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.7 2024-05-09T20:27:12.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.7 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.8 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.8 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.9 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.9 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.10 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.10 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.11 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1.11 2024-05-09T20:27:12.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.660+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.0 2024-05-09T20:27:12.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.664+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1 2024-05-09T20:27:12.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.664+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2 2024-05-09T20:27:12.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.664+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.3 2024-05-09T20:27:12.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.664+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.4 2024-05-09T20:27:12.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.664+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.5 2024-05-09T20:27:12.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.668+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.6 2024-05-09T20:27:12.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.668+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.7 2024-05-09T20:27:12.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.668+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.8 2024-05-09T20:27:12.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.672+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.9 2024-05-09T20:27:12.663 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.672+0000 7f01b4831980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.10 2024-05-09T20:27:12.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.680+0000 7f01b4831980 -1 Injecting error=-5 at location=commit_target_layout 2024-05-09T20:27:12.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.680+0000 7f01b4831980 0 ERROR: commit_reshard failed to commit target index layout: (5) Input/output error 2024-05-09T20:27:12.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.688+0000 7f0068fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:12.683 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.692+0000 7f01b4831980 20 remove_watcher() i=0 2024-05-09T20:27:12.684 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.692+0000 7f01b4831980 2 removed watcher, disabling cache 2024-05-09T20:27:12.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.696+0000 7f01b4831980 20 remove_watcher() i=1 2024-05-09T20:27:12.688 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.696+0000 7f01b4831980 20 remove_watcher() i=2 2024-05-09T20:27:12.690 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.700+0000 7f01b4831980 20 remove_watcher() i=3 2024-05-09T20:27:12.692 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.700+0000 7f01b4831980 20 remove_watcher() i=4 2024-05-09T20:27:12.694 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.704+0000 7f01b4831980 20 remove_watcher() i=5 2024-05-09T20:27:12.696 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.704+0000 7f01b4831980 20 remove_watcher() i=6 2024-05-09T20:27:12.697 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.708+0000 7f01b4831980 20 remove_watcher() i=7 2024-05-09T20:27:12.713 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-09T20:27:12.745 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:12.745 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:12.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 20 realm 2024-05-09T20:27:12.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:12.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:12.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.800+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.804+0000 7f6df9b98980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:12.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.804+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:12.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.804+0000 7f6df9b98980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:12.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.804+0000 7f6df9b98980 20 searching for the correct realm 2024-05-09T20:27:12.798 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:12.808+0000 7f6df76ac640 0 -- 172.21.15.63:0/440702039 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f6df0015070 msgr2=0x7f6df0055630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:13.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.024+0000 7f6df76ac640 0 -- 172.21.15.63:0/440702039 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f6df0015070 msgr2=0x7f6df0055630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:13.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:13.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:13.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:13.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:13.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:13.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:13.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:13.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.036+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:13.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 zone default found 2024-05-09T20:27:13.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 4 Realm: () 2024-05-09T20:27:13.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:13.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:13.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:13.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 zonegroup default 2024-05-09T20:27:13.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 started sync module instance, tier type = 2024-05-09T20:27:13.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.040+0000 7f6df9b98980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:13.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.064+0000 7f6df9b98980 20 add_watcher() i=0 2024-05-09T20:27:13.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.064+0000 7f6df9b98980 20 add_watcher() i=1 2024-05-09T20:27:13.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.064+0000 7f6df9b98980 20 add_watcher() i=2 2024-05-09T20:27:13.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.064+0000 7f6df9b98980 20 add_watcher() i=3 2024-05-09T20:27:13.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.064+0000 7f6df9b98980 20 add_watcher() i=4 2024-05-09T20:27:13.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.064+0000 7f6df9b98980 20 add_watcher() i=5 2024-05-09T20:27:13.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.064+0000 7f6df9b98980 20 add_watcher() i=6 2024-05-09T20:27:13.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.068+0000 7f6df9b98980 20 add_watcher() i=7 2024-05-09T20:27:13.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.068+0000 7f6df9b98980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:13.063 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.072+0000 7f6cadfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:13.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.072+0000 7f6df9b98980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:13.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.072+0000 7f6df9b98980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:13.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.072+0000 7f6df9b98980 5 note: GC not initialized 2024-05-09T20:27:13.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.072+0000 7f6cacfe1640 20 reqs_thread_entry: start 2024-05-09T20:27:13.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.072+0000 7f6df9b98980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:13.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.072+0000 7f6df9b98980 20 Filter name: none 2024-05-09T20:27:13.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.072+0000 7f6c267fc640 20 reqs_thread_entry: start 2024-05-09T20:27:13.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.076+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.076+0000 7f6df9b98980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:13.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.076+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.076+0000 7f6df9b98980 20 rados_obj.operate() r=0 bl.length=443 2024-05-09T20:27:13.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.076+0000 7f6df9b98980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:13.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.076+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.076+0000 7f6df9b98980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:13.069 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.076+0000 7f6df9b98980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.080+0000 7f6df9b98980 20 rados_obj.operate() r=0 bl.length=443 2024-05-09T20:27:13.070 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.080+0000 7f6df9b98980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:13.073 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.080+0000 7f6cadfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:13.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.088+0000 7f6df9b98980 20 remove_watcher() i=0 2024-05-09T20:27:13.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.088+0000 7f6df9b98980 2 removed watcher, disabling cache 2024-05-09T20:27:13.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.088+0000 7f6df9b98980 20 remove_watcher() i=1 2024-05-09T20:27:13.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.092+0000 7f6df9b98980 20 remove_watcher() i=2 2024-05-09T20:27:13.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.092+0000 7f6df9b98980 20 remove_watcher() i=3 2024-05-09T20:27:13.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.096+0000 7f6df9b98980 20 remove_watcher() i=4 2024-05-09T20:27:13.088 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.096+0000 7f6df9b98980 20 remove_watcher() i=5 2024-05-09T20:27:13.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.100+0000 7f6df9b98980 20 remove_watcher() i=6 2024-05-09T20:27:13.091 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.100+0000 7f6df9b98980 20 remove_watcher() i=7 2024-05-09T20:27:13.107 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:13.107 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:13.107 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-commit-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:27:13.107 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:13.219 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:27:13.255 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:13.255 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:13.301 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.308+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 realm 2024-05-09T20:27:13.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:13.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.303 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:13.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.312+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.316+0000 7f22aef77980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:13.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.316+0000 7f22aef77980 20 searching for the correct realm 2024-05-09T20:27:13.308 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.316+0000 7f22ad28c640 0 -- 172.21.15.63:0/236905977 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f22a003f060 msgr2=0x7f22a0035990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:13.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.528+0000 7f22ac28a640 0 -- 172.21.15.63:0/236905977 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f229c01a040 msgr2=0x7f229c00e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:13.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:13.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:13.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:13.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:13.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:13.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:13.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.544+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:13.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:13.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 zone default found 2024-05-09T20:27:13.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 4 Realm: () 2024-05-09T20:27:13.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:13.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:13.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:13.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 zonegroup default 2024-05-09T20:27:13.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 started sync module instance, tier type = 2024-05-09T20:27:13.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.548+0000 7f22aef77980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:13.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.572+0000 7f22aef77980 20 add_watcher() i=0 2024-05-09T20:27:13.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.572+0000 7f22aef77980 20 add_watcher() i=1 2024-05-09T20:27:13.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.572+0000 7f22aef77980 20 add_watcher() i=2 2024-05-09T20:27:13.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.572+0000 7f22aef77980 20 add_watcher() i=3 2024-05-09T20:27:13.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.572+0000 7f22aef77980 20 add_watcher() i=4 2024-05-09T20:27:13.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.572+0000 7f22aef77980 20 add_watcher() i=5 2024-05-09T20:27:13.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.572+0000 7f22aef77980 20 add_watcher() i=6 2024-05-09T20:27:13.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.576+0000 7f22aef77980 20 add_watcher() i=7 2024-05-09T20:27:13.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.576+0000 7f22aef77980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:13.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f2162fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:13.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f22aef77980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:13.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f22aef77980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:13.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f22aef77980 5 note: GC not initialized 2024-05-09T20:27:13.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f2161fe3640 20 reqs_thread_entry: start 2024-05-09T20:27:13.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f22aef77980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:13.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f22aef77980 20 Filter name: none 2024-05-09T20:27:13.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f20d37fe640 20 reqs_thread_entry: start 2024-05-09T20:27:13.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f22aef77980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-05-09T20:27:13.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.580+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:13.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-05-09T20:27:13.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-05-09T20:27:13.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 : miss 2024-05-09T20:27:13.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 20 rados_obj.operate() r=0 bl.length=443 2024-05-09T20:27:13.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 info.flags=0x17 2024-05-09T20:27:13.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 to cache LRU end 2024-05-09T20:27:13.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:13.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:13.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.584+0000 7f22aef77980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 2024-05-09T20:27:13.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.588+0000 7f22aef77980 10 init_target_layout removing existing target index objects from a previous reshard attempt 2024-05-09T20:27:13.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.604+0000 7f22aef77980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 info.flags=0x17 2024-05-09T20:27:13.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.604+0000 7f22aef77980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 to cache LRU end 2024-05-09T20:27:13.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.604+0000 7f22aef77980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:13.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.604+0000 7f22aef77980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24, ofs0, ns] 2024-05-09T20:27:13.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.604+0000 7f2183fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431590 cookie 94031394282608 notifier 6314 bl.length()=923 2024-05-09T20:27:13.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.608+0000 7f2183fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 info.flags=0x17 2024-05-09T20:27:13.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.608+0000 7f2183fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 to cache LRU end 2024-05-09T20:27:13.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.608+0000 7f2183fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:13.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.608+0000 7f22aef77980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): orig_sources= new_sources= 2024-05-09T20:27:13.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.608+0000 7f22aef77980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): potential sources added=[] removed=[] 2024-05-09T20:27:13.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.608+0000 7f22aef77980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): orig_dests= new_dests= 2024-05-09T20:27:13.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.608+0000 7f22aef77980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): potential dests added=[] removed=[] 2024-05-09T20:27:13.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.0 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.0 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.1 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.1 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.2 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.2 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.3 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.3 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.4 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.4 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.5 2024-05-09T20:27:13.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.5 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.6 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.6 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.7 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.7 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.8 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.612+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.8 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.9 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.9 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.10 2024-05-09T20:27:13.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.10 2024-05-09T20:27:13.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.11 2024-05-09T20:27:13.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2.11 2024-05-09T20:27:13.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.0 2024-05-09T20:27:13.608 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.1 2024-05-09T20:27:13.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.2 2024-05-09T20:27:13.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.3 2024-05-09T20:27:13.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.4 2024-05-09T20:27:13.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.616+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.5 2024-05-09T20:27:13.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.620+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.6 2024-05-09T20:27:13.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.620+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.7 2024-05-09T20:27:13.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.620+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.8 2024-05-09T20:27:13.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.620+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.9 2024-05-09T20:27:13.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.624+0000 7f22aef77980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24.10 2024-05-09T20:27:13.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.632+0000 7f22aef77980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 : hit (requested=0x17, cached=0x17) 2024-05-09T20:27:13.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.632+0000 7f22aef77980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 2024-05-09T20:27:13.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f22aef77980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 info.flags=0x17 2024-05-09T20:27:13.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f22aef77980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 to cache LRU end 2024-05-09T20:27:13.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f22aef77980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:13.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f22aef77980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24, ofs0, ns] 2024-05-09T20:27:13.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f2183fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431591 cookie 94031394282608 notifier 6314 bl.length()=891 2024-05-09T20:27:13.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f2183fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 info.flags=0x17 2024-05-09T20:27:13.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f2183fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 to cache LRU end 2024-05-09T20:27:13.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f2183fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:13.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f22aef77980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): orig_sources= new_sources= 2024-05-09T20:27:13.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f22aef77980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): potential sources added=[] removed=[] 2024-05-09T20:27:13.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f22aef77980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): orig_dests= new_dests= 2024-05-09T20:27:13.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.636+0000 7f22aef77980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24]): potential dests added=[] removed=[] 2024-05-09T20:27:13.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.644+0000 7f22aef77980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-05-09T20:27:13.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.648+0000 7f2162fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:13.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.652+0000 7f22aef77980 20 remove_watcher() i=0 2024-05-09T20:27:13.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.652+0000 7f22aef77980 2 removed watcher, disabling cache 2024-05-09T20:27:13.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.656+0000 7f22aef77980 20 remove_watcher() i=1 2024-05-09T20:27:13.649 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.656+0000 7f22aef77980 20 remove_watcher() i=2 2024-05-09T20:27:13.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.660+0000 7f22aef77980 20 remove_watcher() i=3 2024-05-09T20:27:13.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.664+0000 7f22aef77980 20 remove_watcher() i=4 2024-05-09T20:27:13.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.664+0000 7f22aef77980 20 remove_watcher() i=5 2024-05-09T20:27:13.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.664+0000 7f22aef77980 20 remove_watcher() i=6 2024-05-09T20:27:13.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.668+0000 7f22aef77980 20 remove_watcher() i=7 2024-05-09T20:27:13.677 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-09T20:27:13.709 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:13.710 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:13.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 20 realm 2024-05-09T20:27:13.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:13.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.764+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.768+0000 7fe48faa1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:13.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.768+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.768+0000 7fe48faa1980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:13.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.768+0000 7fe48faa1980 20 searching for the correct realm 2024-05-09T20:27:13.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.772+0000 7fe48ddb6640 0 -- 172.21.15.63:0/3259896905 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fe48404f030 msgr2=0x7fe48404f920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:13.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:13.984+0000 7fe48ddb6640 0 -- 172.21.15.63:0/3259896905 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fe48404f030 msgr2=0x7fe48404f920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:13.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:13.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:13.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:13.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:13.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:13.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:13.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:13.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:13.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 zone default found 2024-05-09T20:27:13.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 4 Realm: () 2024-05-09T20:27:13.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:13.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:13.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:13.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 zonegroup default 2024-05-09T20:27:13.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.004+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.008+0000 7fe48faa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.008+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:13.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.008+0000 7fe48faa1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:13.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.008+0000 7fe48faa1980 20 started sync module instance, tier type = 2024-05-09T20:27:13.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.008+0000 7fe48faa1980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:14.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.032+0000 7fe48faa1980 20 add_watcher() i=0 2024-05-09T20:27:14.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.032+0000 7fe48faa1980 20 add_watcher() i=1 2024-05-09T20:27:14.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.032+0000 7fe48faa1980 20 add_watcher() i=2 2024-05-09T20:27:14.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.032+0000 7fe48faa1980 20 add_watcher() i=3 2024-05-09T20:27:14.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.032+0000 7fe48faa1980 20 add_watcher() i=4 2024-05-09T20:27:14.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.032+0000 7fe48faa1980 20 add_watcher() i=5 2024-05-09T20:27:14.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.032+0000 7fe48faa1980 20 add_watcher() i=6 2024-05-09T20:27:14.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.036+0000 7fe48faa1980 20 add_watcher() i=7 2024-05-09T20:27:14.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.036+0000 7fe48faa1980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:14.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe343fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:14.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe48faa1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:14.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe48faa1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:14.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe48faa1980 5 note: GC not initialized 2024-05-09T20:27:14.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe342fe5640 20 reqs_thread_entry: start 2024-05-09T20:27:14.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe48faa1980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:14.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe48faa1980 20 Filter name: none 2024-05-09T20:27:14.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe340fe1640 20 reqs_thread_entry: start 2024-05-09T20:27:14.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.040+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:14.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.044+0000 7fe48faa1980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:14.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.044+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:14.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.044+0000 7fe48faa1980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:14.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.044+0000 7fe48faa1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:14.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.044+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:14.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.044+0000 7fe48faa1980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:14.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.044+0000 7fe48faa1980 20 rados->read ofs=0 len=0 2024-05-09T20:27:14.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.048+0000 7fe48faa1980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:14.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.048+0000 7fe48faa1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:14.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.048+0000 7fe343fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:14.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.052+0000 7fe48faa1980 20 remove_watcher() i=0 2024-05-09T20:27:14.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.052+0000 7fe48faa1980 2 removed watcher, disabling cache 2024-05-09T20:27:14.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.056+0000 7fe48faa1980 20 remove_watcher() i=1 2024-05-09T20:27:14.050 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.060+0000 7fe48faa1980 20 remove_watcher() i=2 2024-05-09T20:27:14.052 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.060+0000 7fe48faa1980 20 remove_watcher() i=3 2024-05-09T20:27:14.055 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.064+0000 7fe48faa1980 20 remove_watcher() i=4 2024-05-09T20:27:14.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.064+0000 7fe48faa1980 20 remove_watcher() i=5 2024-05-09T20:27:14.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.068+0000 7fe48faa1980 20 remove_watcher() i=6 2024-05-09T20:27:14.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:14.068+0000 7fe48faa1980 20 remove_watcher() i=7 2024-05-09T20:27:14.077 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:14.077 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:14.077 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-commit-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.24 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-09T20:27:14.077 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:14.950 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with ECANCELED injected at commit_target_layout 2024-05-09T20:27:14.951 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:17.077 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-09T20:27:17.111 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:17.111 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:17.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.164+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 realm 2024-05-09T20:27:17.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:17.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.159 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:17.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.168+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.172+0000 7fc2bae64980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:17.162 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.172+0000 7fc2bae64980 20 searching for the correct realm 2024-05-09T20:27:17.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.380+0000 7fc2b9179640 0 -- 172.21.15.63:0/4184195019 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fc2ac008040 msgr2=0x7fc2ac05c890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:17.379 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.388+0000 7fc2b9179640 0 -- 172.21.15.63:0/4184195019 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fc184077410 msgr2=0x7fc184079840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:17.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:17.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:17.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:17.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:17.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:17.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:17.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.394 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:17.395 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:17.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 zone default found 2024-05-09T20:27:17.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 4 Realm: () 2024-05-09T20:27:17.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:17.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:17.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:17.396 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 zonegroup default 2024-05-09T20:27:17.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.404+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.397 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.408+0000 7fc2bae64980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.408+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.408+0000 7fc2bae64980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.408+0000 7fc2bae64980 20 started sync module instance, tier type = 2024-05-09T20:27:17.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.408+0000 7fc2bae64980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:17.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.432+0000 7fc2bae64980 20 add_watcher() i=0 2024-05-09T20:27:17.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.432+0000 7fc2bae64980 20 add_watcher() i=1 2024-05-09T20:27:17.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.432+0000 7fc2bae64980 20 add_watcher() i=2 2024-05-09T20:27:17.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.432+0000 7fc2bae64980 20 add_watcher() i=3 2024-05-09T20:27:17.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.432+0000 7fc2bae64980 20 add_watcher() i=4 2024-05-09T20:27:17.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.432+0000 7fc2bae64980 20 add_watcher() i=5 2024-05-09T20:27:17.423 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.432+0000 7fc2bae64980 20 add_watcher() i=6 2024-05-09T20:27:17.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.436+0000 7fc2bae64980 20 add_watcher() i=7 2024-05-09T20:27:17.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.436+0000 7fc2bae64980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:17.430 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc16efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:17.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc2bae64980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:17.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc2bae64980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:17.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc2bae64980 5 note: GC not initialized 2024-05-09T20:27:17.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc16dfe3640 20 reqs_thread_entry: start 2024-05-09T20:27:17.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc2bae64980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:17.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc2bae64980 20 Filter name: none 2024-05-09T20:27:17.432 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc0df7fe640 20 reqs_thread_entry: start 2024-05-09T20:27:17.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.440+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.433 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.444+0000 7fc2bae64980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:17.434 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.444+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.444+0000 7fc2bae64980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:17.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.444+0000 7fc2bae64980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:17.435 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.444+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.444+0000 7fc2bae64980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:17.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.444+0000 7fc2bae64980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.448+0000 7fc2bae64980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:17.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.448+0000 7fc2bae64980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:17.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.452+0000 7fc16efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:17.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.456+0000 7fc2bae64980 20 remove_watcher() i=0 2024-05-09T20:27:17.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.456+0000 7fc2bae64980 2 removed watcher, disabling cache 2024-05-09T20:27:17.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.456+0000 7fc2bae64980 20 remove_watcher() i=1 2024-05-09T20:27:17.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.460+0000 7fc2bae64980 20 remove_watcher() i=2 2024-05-09T20:27:17.453 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.464+0000 7fc2bae64980 20 remove_watcher() i=3 2024-05-09T20:27:17.456 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.464+0000 7fc2bae64980 20 remove_watcher() i=4 2024-05-09T20:27:17.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.468+0000 7fc2bae64980 20 remove_watcher() i=5 2024-05-09T20:27:17.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.468+0000 7fc2bae64980 20 remove_watcher() i=6 2024-05-09T20:27:17.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.472+0000 7fc2bae64980 20 remove_watcher() i=7 2024-05-09T20:27:17.477 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:17.478 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:17.478 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-commit-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:27:17.478 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:17.478 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at commit_target_layout --inject-error-code 125 2024-05-09T20:27:17.519 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:17.519 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:17.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 20 realm 2024-05-09T20:27:17.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:17.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.572+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.576+0000 7f5251b59980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:17.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.576+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.576+0000 7f5251b59980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:17.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.576+0000 7f5251b59980 20 searching for the correct realm 2024-05-09T20:27:17.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.580+0000 7f524fe6e640 0 -- 172.21.15.63:0/1589042258 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f5238025070 msgr2=0x7f523803c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:17.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.584+0000 7f524ee6c640 0 -- 172.21.15.63:0/1589042258 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f52480fb120 msgr2=0x7f52480ce410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:17.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:17.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:17.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:17.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:17.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:17.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:17.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:17.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:17.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 zone default found 2024-05-09T20:27:17.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 4 Realm: () 2024-05-09T20:27:17.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:17.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:17.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:17.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 zonegroup default 2024-05-09T20:27:17.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.596+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.600+0000 7f5251b59980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.600+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.600+0000 7f5251b59980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.600+0000 7f5251b59980 20 started sync module instance, tier type = 2024-05-09T20:27:17.591 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.600+0000 7f5251b59980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:17.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 20 add_watcher() i=0 2024-05-09T20:27:17.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 20 add_watcher() i=1 2024-05-09T20:27:17.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 20 add_watcher() i=2 2024-05-09T20:27:17.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 20 add_watcher() i=3 2024-05-09T20:27:17.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 20 add_watcher() i=4 2024-05-09T20:27:17.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 20 add_watcher() i=5 2024-05-09T20:27:17.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 20 add_watcher() i=6 2024-05-09T20:27:17.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 20 add_watcher() i=7 2024-05-09T20:27:17.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.616+0000 7f5251b59980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:17.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5105fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:17.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:17.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:17.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 5 note: GC not initialized 2024-05-09T20:27:17.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5104fe1640 20 reqs_thread_entry: start 2024-05-09T20:27:17.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:17.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 20 Filter name: none 2024-05-09T20:27:17.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f507e7fc640 20 reqs_thread_entry: start 2024-05-09T20:27:17.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 10 cache get: name=default.rgw.meta+root+error-at-commit-target-layout : miss 2024-05-09T20:27:17.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:17.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 10 cache put: name=default.rgw.meta+root+error-at-commit-target-layout info.flags=0x11 2024-05-09T20:27:17.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 10 adding default.rgw.meta+root+error-at-commit-target-layout to cache LRU end 2024-05-09T20:27:17.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 : miss 2024-05-09T20:27:17.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.620+0000 7f5251b59980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.624+0000 7f5251b59980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:17.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.624+0000 7f5251b59980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 info.flags=0x17 2024-05-09T20:27:17.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.624+0000 7f5251b59980 10 adding default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 to cache LRU end 2024-05-09T20:27:17.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.624+0000 7f5251b59980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:17.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.624+0000 7f5251b59980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:17.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.624+0000 7f5251b59980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 2024-05-09T20:27:17.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5251b59980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 info.flags=0x17 2024-05-09T20:27:17.625 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5251b59980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 to cache LRU end 2024-05-09T20:27:17.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5251b59980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:17.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5251b59980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25, ofs0, ns] 2024-05-09T20:27:17.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5132ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431594 cookie 94413169908336 notifier 6368 bl.length()=923 2024-05-09T20:27:17.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5132ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 info.flags=0x17 2024-05-09T20:27:17.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5132ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 to cache LRU end 2024-05-09T20:27:17.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5132ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:17.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5251b59980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25]): orig_sources= new_sources= 2024-05-09T20:27:17.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5251b59980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25]): potential sources added=[] removed=[] 2024-05-09T20:27:17.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5251b59980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25]): orig_dests= new_dests= 2024-05-09T20:27:17.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.636+0000 7f5251b59980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25]): potential dests added=[] removed=[] 2024-05-09T20:27:17.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.0 2024-05-09T20:27:17.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.0 2024-05-09T20:27:17.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.1 2024-05-09T20:27:17.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.1 2024-05-09T20:27:17.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.2 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.2 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.3 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.3 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.4 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.4 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.5 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.640+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.5 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.6 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.6 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.7 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.7 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.8 2024-05-09T20:27:17.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.8 2024-05-09T20:27:17.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.9 2024-05-09T20:27:17.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.9 2024-05-09T20:27:17.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.10 2024-05-09T20:27:17.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.10 2024-05-09T20:27:17.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.11 2024-05-09T20:27:17.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1.11 2024-05-09T20:27:17.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.0 2024-05-09T20:27:17.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.1 2024-05-09T20:27:17.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.2 2024-05-09T20:27:17.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.3 2024-05-09T20:27:17.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.644+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.4 2024-05-09T20:27:17.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.648+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.5 2024-05-09T20:27:17.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.648+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.6 2024-05-09T20:27:17.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.648+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.7 2024-05-09T20:27:17.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.648+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.8 2024-05-09T20:27:17.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.648+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.9 2024-05-09T20:27:17.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.652+0000 7f5251b59980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25.10 2024-05-09T20:27:17.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.656+0000 7f5251b59980 -1 Injecting error=-125 at location=commit_target_layout 2024-05-09T20:27:17.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.656+0000 7f5251b59980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 : hit (requested=0x17, cached=0x17) 2024-05-09T20:27:17.648 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.656+0000 7f5251b59980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 2024-05-09T20:27:17.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5251b59980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 info.flags=0x17 2024-05-09T20:27:17.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5251b59980 10 moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 to cache LRU end 2024-05-09T20:27:17.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5251b59980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:17.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5251b59980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25, ofs0, ns] 2024-05-09T20:27:17.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5132ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431595 cookie 94413169908336 notifier 6368 bl.length()=891 2024-05-09T20:27:17.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5132ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 info.flags=0x17 2024-05-09T20:27:17.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5132ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 to cache LRU end 2024-05-09T20:27:17.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5132ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:17.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5251b59980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25]): orig_sources= new_sources= 2024-05-09T20:27:17.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5251b59980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25]): potential sources added=[] removed=[] 2024-05-09T20:27:17.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5251b59980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25]): orig_dests= new_dests= 2024-05-09T20:27:17.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.660+0000 7f5251b59980 20 handle_bi_update(): bucket=:error-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25]): potential dests added=[] removed=[] 2024-05-09T20:27:17.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.668+0000 7f5251b59980 1 execute INFO: reshard of bucket "error-at-commit-target-layout" completed successfully 2024-05-09T20:27:17.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.668+0000 7f5105fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:17.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.676+0000 7f5251b59980 20 remove_watcher() i=0 2024-05-09T20:27:17.665 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.676+0000 7f5251b59980 2 removed watcher, disabling cache 2024-05-09T20:27:17.667 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.676+0000 7f5251b59980 20 remove_watcher() i=1 2024-05-09T20:27:17.669 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.680+0000 7f5251b59980 20 remove_watcher() i=2 2024-05-09T20:27:17.671 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.680+0000 7f5251b59980 20 remove_watcher() i=3 2024-05-09T20:27:17.673 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.684+0000 7f5251b59980 20 remove_watcher() i=4 2024-05-09T20:27:17.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.684+0000 7f5251b59980 20 remove_watcher() i=5 2024-05-09T20:27:17.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.684+0000 7f5251b59980 20 remove_watcher() i=6 2024-05-09T20:27:17.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.688+0000 7f5251b59980 20 remove_watcher() i=7 2024-05-09T20:27:17.691 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-commit-target-layout 2024-05-09T20:27:17.718 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:17.718 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:17.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.768+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 realm 2024-05-09T20:27:17.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:17.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:17.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:17.765 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.772+0000 7f6359225980 20 searching for the correct realm 2024-05-09T20:27:17.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.776+0000 7f6356d39640 0 -- 172.21.15.63:0/1478074329 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f6350049b30 msgr2=0x7f6350030990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:17.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:17.992+0000 7f6356d39640 0 -- 172.21.15.63:0/1478074329 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f6350049b30 msgr2=0x7f6350030990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:17.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:17.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:17.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:17.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:17.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:17.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:17.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.995 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:17.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:17.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 zone default found 2024-05-09T20:27:17.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 4 Realm: () 2024-05-09T20:27:17.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:17.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:17.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:17.997 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 zonegroup default 2024-05-09T20:27:17.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.004+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.008+0000 7f6359225980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.008+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.008+0000 7f6359225980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.008+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:17.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.008+0000 7f6359225980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:17.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.008+0000 7f6359225980 20 started sync module instance, tier type = 2024-05-09T20:27:17.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.008+0000 7f6359225980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:18.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.032+0000 7f6359225980 20 add_watcher() i=0 2024-05-09T20:27:18.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.032+0000 7f6359225980 20 add_watcher() i=1 2024-05-09T20:27:18.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.032+0000 7f6359225980 20 add_watcher() i=2 2024-05-09T20:27:18.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.032+0000 7f6359225980 20 add_watcher() i=3 2024-05-09T20:27:18.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.032+0000 7f6359225980 20 add_watcher() i=4 2024-05-09T20:27:18.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.032+0000 7f6359225980 20 add_watcher() i=5 2024-05-09T20:27:18.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.032+0000 7f6359225980 20 add_watcher() i=6 2024-05-09T20:27:18.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.036+0000 7f6359225980 20 add_watcher() i=7 2024-05-09T20:27:18.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.036+0000 7f6359225980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:18.031 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f620d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:18.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f6359225980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:18.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f6359225980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:18.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f6359225980 5 note: GC not initialized 2024-05-09T20:27:18.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f6187fff640 20 reqs_thread_entry: start 2024-05-09T20:27:18.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f6359225980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:18.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f6359225980 20 Filter name: none 2024-05-09T20:27:18.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f6185ffb640 20 reqs_thread_entry: start 2024-05-09T20:27:18.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.040+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:18.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.044+0000 7f6359225980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:18.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.044+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:18.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.044+0000 7f6359225980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:18.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.044+0000 7f6359225980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:18.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.044+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:18.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.044+0000 7f6359225980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:18.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.044+0000 7f6359225980 20 rados->read ofs=0 len=0 2024-05-09T20:27:18.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.048+0000 7f6359225980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:18.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.048+0000 7f6359225980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:18.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.052+0000 7f620d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:18.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.056+0000 7f6359225980 20 remove_watcher() i=0 2024-05-09T20:27:18.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.056+0000 7f6359225980 2 removed watcher, disabling cache 2024-05-09T20:27:18.049 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.060+0000 7f6359225980 20 remove_watcher() i=1 2024-05-09T20:27:18.051 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.060+0000 7f6359225980 20 remove_watcher() i=2 2024-05-09T20:27:18.053 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.064+0000 7f6359225980 20 remove_watcher() i=3 2024-05-09T20:27:18.056 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.064+0000 7f6359225980 20 remove_watcher() i=4 2024-05-09T20:27:18.058 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.068+0000 7f6359225980 20 remove_watcher() i=5 2024-05-09T20:27:18.059 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.068+0000 7f6359225980 20 remove_watcher() i=6 2024-05-09T20:27:18.061 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:18.068+0000 7f6359225980 20 remove_watcher() i=7 2024-05-09T20:27:18.077 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:18.077 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:18.077 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-commit-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.25 num_objects: 20 size_kb: 1 num_shards: 12 2024-05-09T20:27:18.077 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:18.960 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with abort at commit_target_layout 2024-05-09T20:27:18.961 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.105 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-05-09T20:27:21.140 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:21.140 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:21.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.192+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 realm 2024-05-09T20:27:21.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:21.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.196+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.200+0000 7f1fd924e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:21.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.200+0000 7f1fd924e980 20 searching for the correct realm 2024-05-09T20:27:21.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.204+0000 7f1fd6d62640 0 -- 172.21.15.63:0/1359564317 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f1fd001dc10 msgr2=0x7f1fd0024900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:21.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.420+0000 7f1fd6d62640 0 -- 172.21.15.63:0/1359564317 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f1fd001dc10 msgr2=0x7f1fd0024900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:21.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:21.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:21.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:21.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:21.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:21.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:21.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.425 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.432+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:21.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 zone default found 2024-05-09T20:27:21.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 4 Realm: () 2024-05-09T20:27:21.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:21.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:21.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:21.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 zonegroup default 2024-05-09T20:27:21.427 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 started sync module instance, tier type = 2024-05-09T20:27:21.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.436+0000 7f1fd924e980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:21.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.460+0000 7f1fd924e980 20 add_watcher() i=0 2024-05-09T20:27:21.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.460+0000 7f1fd924e980 20 add_watcher() i=1 2024-05-09T20:27:21.451 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.460+0000 7f1fd924e980 20 add_watcher() i=2 2024-05-09T20:27:21.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.460+0000 7f1fd924e980 20 add_watcher() i=3 2024-05-09T20:27:21.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.460+0000 7f1fd924e980 20 add_watcher() i=4 2024-05-09T20:27:21.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.460+0000 7f1fd924e980 20 add_watcher() i=5 2024-05-09T20:27:21.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.460+0000 7f1fd924e980 20 add_watcher() i=6 2024-05-09T20:27:21.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.464+0000 7f1fd924e980 20 add_watcher() i=7 2024-05-09T20:27:21.454 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.464+0000 7f1fd924e980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:21.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1e8d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:21.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1fd924e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:21.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1fd924e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:21.459 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1fd924e980 5 note: GC not initialized 2024-05-09T20:27:21.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1e07fff640 20 reqs_thread_entry: start 2024-05-09T20:27:21.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1fd924e980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:21.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1fd924e980 20 Filter name: none 2024-05-09T20:27:21.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1e05ffb640 20 reqs_thread_entry: start 2024-05-09T20:27:21.461 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.468+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:21.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:21.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:21.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:21.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:21.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.472+0000 7f1fd924e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:21.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.476+0000 7f1e8d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:21.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.480+0000 7f1fd924e980 20 remove_watcher() i=0 2024-05-09T20:27:21.473 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.480+0000 7f1fd924e980 2 removed watcher, disabling cache 2024-05-09T20:27:21.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.484+0000 7f1fd924e980 20 remove_watcher() i=1 2024-05-09T20:27:21.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.488+0000 7f1fd924e980 20 remove_watcher() i=2 2024-05-09T20:27:21.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.488+0000 7f1fd924e980 20 remove_watcher() i=3 2024-05-09T20:27:21.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.492+0000 7f1fd924e980 20 remove_watcher() i=4 2024-05-09T20:27:21.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.492+0000 7f1fd924e980 20 remove_watcher() i=5 2024-05-09T20:27:21.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.496+0000 7f1fd924e980 20 remove_watcher() i=6 2024-05-09T20:27:21.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.496+0000 7f1fd924e980 20 remove_watcher() i=7 2024-05-09T20:27:21.503 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.503 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:21.503 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-commit-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:27:21.503 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.504 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at commit_target_layout 2024-05-09T20:27:21.542 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:21.542 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:21.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 20 realm 2024-05-09T20:27:21.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:21.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.589 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:21.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.600+0000 7f19987c5980 20 searching for the correct realm 2024-05-09T20:27:21.593 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.604+0000 7f1996ada640 0 -- 172.21.15.63:0/1395419920 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f198c0a58f0 msgr2=0x7f198c110890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:21.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.608+0000 7f1995ad8640 0 -- 172.21.15.63:0/1395419920 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f1988008030 msgr2=0x7f198802c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:21.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:21.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:21.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:21.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:21.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:21.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:21.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 zone default found 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 4 Realm: () 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 zonegroup default 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 started sync module instance, tier type = 2024-05-09T20:27:21.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.620+0000 7f19987c5980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:21.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=0 2024-05-09T20:27:21.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=1 2024-05-09T20:27:21.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=2 2024-05-09T20:27:21.628 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=3 2024-05-09T20:27:21.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=4 2024-05-09T20:27:21.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=5 2024-05-09T20:27:21.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=6 2024-05-09T20:27:21.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 20 add_watcher() i=7 2024-05-09T20:27:21.629 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:21.632 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f184d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 5 note: GC not initialized 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f18477fe640 20 reqs_thread_entry: start 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 20 Filter name: none 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f18457fa640 20 reqs_thread_entry: start 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-05-09T20:27:21.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.640+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:21.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-05-09T20:27:21.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-05-09T20:27:21.634 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 : miss 2024-05-09T20:27:21.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:21.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:21.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.644+0000 7f19987c5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 2024-05-09T20:27:21.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.652+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.652+0000 7f19987c5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.652+0000 7f19987c5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.645 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.652+0000 7f19987c5980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26, ofs0, ns] 2024-05-09T20:27:21.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431598 cookie 94026911740896 notifier 6801 bl.length()=923 2024-05-09T20:27:21.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.646 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_sources= new_sources= 2024-05-09T20:27:21.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential sources added=[] removed=[] 2024-05-09T20:27:21.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_dests= new_dests= 2024-05-09T20:27:21.647 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential dests added=[] removed=[] 2024-05-09T20:27:21.651 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.0 2024-05-09T20:27:21.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.0 2024-05-09T20:27:21.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.1 2024-05-09T20:27:21.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.1 2024-05-09T20:27:21.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.2 2024-05-09T20:27:21.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.2 2024-05-09T20:27:21.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.3 2024-05-09T20:27:21.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.3 2024-05-09T20:27:21.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.4 2024-05-09T20:27:21.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.4 2024-05-09T20:27:21.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.5 2024-05-09T20:27:21.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.5 2024-05-09T20:27:21.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.6 2024-05-09T20:27:21.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.6 2024-05-09T20:27:21.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.7 2024-05-09T20:27:21.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.7 2024-05-09T20:27:21.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.8 2024-05-09T20:27:21.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.8 2024-05-09T20:27:21.654 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.9 2024-05-09T20:27:21.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.9 2024-05-09T20:27:21.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.10 2024-05-09T20:27:21.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.10 2024-05-09T20:27:21.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.11 2024-05-09T20:27:21.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.11 2024-05-09T20:27:21.655 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.0 2024-05-09T20:27:21.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1 2024-05-09T20:27:21.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.2 2024-05-09T20:27:21.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.3 2024-05-09T20:27:21.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.4 2024-05-09T20:27:21.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.5 2024-05-09T20:27:21.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.6 2024-05-09T20:27:21.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.7 2024-05-09T20:27:21.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.8 2024-05-09T20:27:21.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.9 2024-05-09T20:27:21.660 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.10 2024-05-09T20:27:21.665 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f19987c5980 time 2024-05-09T20:27:21.677617+0000 2024-05-09T20:27:21.665 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:27:21.665 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f199c09be25] 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f199c09bfd9] 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55845744eca1] 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x57f) [0x55845745f67f] 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f199af05d90] 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.676+0000 7f19987c5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f19987c5980 time 2024-05-09T20:27:21.677617+0000 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.666 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f199c09be25] 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f199c09bfd9] 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55845744eca1] 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x57f) [0x55845745f67f] 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f199af05d90] 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr:*** Caught signal (Aborted) ** 2024-05-09T20:27:21.667 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f19987c5980 thread_name:radosgw-admin 2024-05-09T20:27:21.684 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f199af1e520] 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f199c09be7f] 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f199c09bfd9] 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55845744eca1] 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x57f) [0x55845745f67f] 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f199af05d90] 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:27:21.685 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.676+0000 7f19987c5980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f19987c5980 thread_name:radosgw-admin 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f199af1e520] 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f199c09be7f] 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f199c09bfd9] 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55845744eca1] 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x57f) [0x55845745f67f] 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:27:21.686 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f199af05d90] 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr:--- begin dump of recent events --- 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -499> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding auth protocol: cephx 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -498> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding auth protocol: cephx 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -497> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding auth protocol: cephx 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -496> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -495> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -494> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -493> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -492> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.687 INFO:tasks.workunit.client.0.smithi063.stderr: -491> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -490> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -489> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -488> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -487> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -486> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -485> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -484> 2024-05-09T20:27:21.576+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -483> 2024-05-09T20:27:21.576+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -482> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 asok(0x5584587bc430) register_command rotate-key hook 0x5584589c5b08 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -481> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _reopen_session rank -1 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -480> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _add_conns ranks=[0,2,1] 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -479> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-a con 0x558458877360 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -478> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-c con 0x558458881d70 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:27:21.688 INFO:tasks.workunit.client.0.smithi063.stderr: -477> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-b con 0x55845890c970 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -476> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -475> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -474> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -473> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): _renew_subs 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -472> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): authenticate will time out at 2048.717521s 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -471> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): get_auth_request con 0x558458877360 auth_method 0 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -470> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -469> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -468> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -467> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): get_auth_request con 0x558458881d70 auth_method 0 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -466> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -465> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -464> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -463> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.689 INFO:tasks.workunit.client.0.smithi063.stderr: -462> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -461> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -460> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -459> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -458> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -457> 2024-05-09T20:27:21.576+0000 7f1996ada640 10 monclient(hunting): get_auth_request con 0x55845890c970 auth_method 0 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -456> 2024-05-09T20:27:21.576+0000 7f1996ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -455> 2024-05-09T20:27:21.576+0000 7f1996ada640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -454> 2024-05-09T20:27:21.576+0000 7f1996ada640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -453> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): handle_auth_done global_id 6407 payload 824 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -452> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient: _finish_hunting 0 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -451> 2024-05-09T20:27:21.576+0000 7f1995ad8640 1 monclient: found mon.noname-c 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -450> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.690 INFO:tasks.workunit.client.0.smithi063.stderr: -449> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr: -448> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr: -447> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: dump: 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr: -446> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: _finish_auth 0 2024-05-09T20:27:21.691 INFO:tasks.workunit.client.0.smithi063.stderr: -445> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: _check_auth_tickets 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr: -444> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: handle_config config(0 keys) 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr: -443> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr: -442> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr: -441> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: dump: 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.692 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -440> 2024-05-09T20:27:21.576+0000 7f1884608640 4 set_mon_vals no callback set 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -439> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 monclient: authenticate success, global_id 6407 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -438> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -437> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -436> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -435> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -434> 2024-05-09T20:27:21.576+0000 7f19987c5980 1 librados: init done 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -433> 2024-05-09T20:27:21.580+0000 7f1882e05640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -432> 2024-05-09T20:27:21.580+0000 7f1882e05640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -431> 2024-05-09T20:27:21.580+0000 7f1882e05640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -430> 2024-05-09T20:27:21.580+0000 7f19962d9640 10 monclient: get_auth_request con 0x7f187c066d60 auth_method 0 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -429> 2024-05-09T20:27:21.580+0000 7f1996ada640 10 monclient: get_auth_request con 0x5584589ed1e0 auth_method 0 2024-05-09T20:27:21.693 INFO:tasks.workunit.client.0.smithi063.stderr: -428> 2024-05-09T20:27:21.580+0000 7f19962d9640 10 monclient: get_auth_request con 0x558458a10640 auth_method 0 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr: -427> 2024-05-09T20:27:21.580+0000 7f1995ad8640 10 monclient: get_auth_request con 0x558458a149a0 auth_method 0 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr: -426> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient: get_auth_request con 0x558458a13ef0 auth_method 0 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr: -425> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: build_initial_monmap 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr: -424> 2024-05-09T20:27:21.584+0000 7f19987c5980 1 build_initial for_mkfs: 0 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr: -423> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: monmap: 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:27:21.694 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -422> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding auth protocol: cephx 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -421> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding auth protocol: cephx 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -420> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding auth protocol: cephx 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -419> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -418> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -417> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -416> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -415> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -414> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -413> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -412> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -411> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -410> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.695 INFO:tasks.workunit.client.0.smithi063.stderr: -409> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -408> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -407> 2024-05-09T20:27:21.584+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -406> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 asok(0x5584587bc430) register_command objecter_requests cmddesc objecter_requests hook 0x558458a22000 EEXIST 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -405> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: init 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -404> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding auth protocol: cephx 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -403> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding auth protocol: cephx 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -402> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding auth protocol: cephx 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -401> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -400> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -399> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -398> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -397> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -396> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -395> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.696 INFO:tasks.workunit.client.0.smithi063.stderr: -394> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -393> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -392> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -391> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -390> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -389> 2024-05-09T20:27:21.584+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -388> 2024-05-09T20:27:21.584+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -387> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 asok(0x5584587bc430) register_command rotate-key cmddesc rotate-key hook 0x558458a1ba60 EEXIST 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -386> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: _reopen_session rank -1 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -385> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -384> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-c con 0x558458a46c60 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -383> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-a con 0x558458a47040 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -382> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-b con 0x558458a42200 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -381> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.697 INFO:tasks.workunit.client.0.smithi063.stderr: -380> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -379> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -378> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): _renew_subs 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -377> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): authenticate will time out at 2048.727502s 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -376> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): get_auth_request con 0x558458a47040 auth_method 0 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -375> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -374> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -373> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -372> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): get_auth_request con 0x558458a46c60 auth_method 0 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -371> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -370> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -369> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -368> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.698 INFO:tasks.workunit.client.0.smithi063.stderr: -367> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -366> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -365> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -364> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -363> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -362> 2024-05-09T20:27:21.584+0000 7f19962d9640 10 monclient(hunting): get_auth_request con 0x558458a42200 auth_method 0 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -361> 2024-05-09T20:27:21.584+0000 7f19962d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -360> 2024-05-09T20:27:21.584+0000 7f19962d9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -359> 2024-05-09T20:27:21.584+0000 7f19962d9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -358> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): handle_auth_done global_id 6410 payload 824 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -357> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient: _finish_hunting 0 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -356> 2024-05-09T20:27:21.584+0000 7f1995ad8640 1 monclient: found mon.noname-c 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -355> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -354> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.699 INFO:tasks.workunit.client.0.smithi063.stderr: -353> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr: -352> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: dump: 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr: -351> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: _finish_auth 0 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr: -350> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: _check_auth_tickets 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr: -349> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: handle_config config(0 keys) 2024-05-09T20:27:21.700 INFO:tasks.workunit.client.0.smithi063.stderr: -348> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr: -347> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr: -346> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: dump: 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr: -345> 2024-05-09T20:27:21.588+0000 7f19972db640 4 set_mon_vals no callback set 2024-05-09T20:27:21.701 INFO:tasks.workunit.client.0.smithi063.stderr: -344> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 monclient: authenticate success, global_id 6410 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -343> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -342> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -341> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -340> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -339> 2024-05-09T20:27:21.588+0000 7f187bfff640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -338> 2024-05-09T20:27:21.588+0000 7f187bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -337> 2024-05-09T20:27:21.588+0000 7f187bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -336> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f1870066ca0 auth_method 0 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -335> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: get_monmap_and_config 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -334> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: build_initial_monmap 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -333> 2024-05-09T20:27:21.588+0000 7f19987c5980 1 build_initial for_mkfs: 0 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr: -332> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: monmap: 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:27:21.702 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr: -331> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr: -330> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr: -329> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr: -328> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr: -327> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.703 INFO:tasks.workunit.client.0.smithi063.stderr: -326> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -325> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -324> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -323> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -322> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -321> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -320> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -319> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -318> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -317> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -316> 2024-05-09T20:27:21.588+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -315> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: init 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -314> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding auth protocol: cephx 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -313> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding auth protocol: cephx 2024-05-09T20:27:21.704 INFO:tasks.workunit.client.0.smithi063.stderr: -312> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding auth protocol: cephx 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -311> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -310> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -309> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -308> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -307> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -306> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -305> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -304> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -303> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -302> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -301> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -300> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -299> 2024-05-09T20:27:21.588+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.705 INFO:tasks.workunit.client.0.smithi063.stderr: -298> 2024-05-09T20:27:21.588+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -297> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 asok(0x5584587bc430) register_command rotate-key cmddesc rotate-key hook 0x7ffdf0b25d38 EEXIST 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -296> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _reopen_session rank -1 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -295> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _add_conns ranks=[2,1,0] 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -294> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-c con 0x558458a98da0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -293> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-b con 0x558458a942b0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -292> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-a con 0x558458a9de90 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -291> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -290> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -289> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -288> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): _renew_subs 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -287> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): authenticate will time out at 2048.731501s 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -286> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): get_auth_request con 0x558458a98da0 auth_method 0 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -285> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.706 INFO:tasks.workunit.client.0.smithi063.stderr: -284> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -283> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -282> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): get_auth_request con 0x558458a9de90 auth_method 0 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -281> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -280> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -279> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -278> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -277> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -276> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -275> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -274> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -273> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -272> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient(hunting): get_auth_request con 0x558458a942b0 auth_method 0 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -271> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.707 INFO:tasks.workunit.client.0.smithi063.stderr: -270> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr: -269> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr: -268> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): handle_auth_done global_id 6413 payload 274 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr: -267> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient: _finish_hunting 0 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr: -266> 2024-05-09T20:27:21.588+0000 7f1995ad8640 1 monclient: found mon.noname-c 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr: -265> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr: -264> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr: -263> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr: -262> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: dump: 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.708 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -261> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: _finish_auth 0 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -260> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: _check_auth_tickets 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -259> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 monclient: authenticate success, global_id 6413 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -258> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: handle_config config(0 keys) 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -257> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: get_monmap_and_config success 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -256> 2024-05-09T20:27:21.592+0000 7f19987c5980 4 set_mon_vals no callback set 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -255> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -254> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr: -253> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: dump: 2024-05-09T20:27:21.709 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr: -252> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: shutdown 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr: -251> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: build_initial_monmap 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr: -250> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 build_initial for_mkfs: 0 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr: -249> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: monmap: 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:27:21.710 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr: -248> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr: -247> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr: -246> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr: -245> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr: -244> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.711 INFO:tasks.workunit.client.0.smithi063.stderr: -243> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -242> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -241> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -240> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -239> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -238> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -237> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -236> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -235> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -234> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -233> 2024-05-09T20:27:21.592+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -232> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: starting msgr at 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -231> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: starting objecter 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -230> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 asok(0x5584587bc430) register_command objecter_requests cmddesc objecter_requests hook 0x558458a941c0 EEXIST 2024-05-09T20:27:21.712 INFO:tasks.workunit.client.0.smithi063.stderr: -229> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: setting wanted keys 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -228> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: calling monclient init 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -227> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: init 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -226> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding auth protocol: cephx 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -225> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding auth protocol: cephx 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -224> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding auth protocol: cephx 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -223> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -222> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -221> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -220> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -219> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -218> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -217> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -216> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -215> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.713 INFO:tasks.workunit.client.0.smithi063.stderr: -214> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -213> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -212> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -211> 2024-05-09T20:27:21.592+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -210> 2024-05-09T20:27:21.592+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -209> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 asok(0x5584587bc430) register_command rotate-key cmddesc rotate-key hook 0x558458a63ac8 EEXIST 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -208> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _reopen_session rank -1 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -207> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -206> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-a con 0x558458a942b0 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -205> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-b con 0x558458a98da0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -204> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-c con 0x558458a9de90 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -203> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -202> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -201> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.714 INFO:tasks.workunit.client.0.smithi063.stderr: -200> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): _renew_subs 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -199> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): authenticate will time out at 2048.734079s 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -198> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): get_auth_request con 0x558458a942b0 auth_method 0 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -197> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -196> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -195> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -194> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): get_auth_request con 0x558458a9de90 auth_method 0 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -193> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -192> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -191> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -190> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -189> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -188> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -187> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.715 INFO:tasks.workunit.client.0.smithi063.stderr: -186> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr: -185> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr: -184> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): handle_auth_done global_id 6801 payload 824 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr: -183> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient: _finish_hunting 0 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr: -182> 2024-05-09T20:27:21.592+0000 7f1995ad8640 1 monclient: found mon.noname-a 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr: -181> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr: -180> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr: -179> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr: -178> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: dump: 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.716 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr: -177> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: _finish_auth 0 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr: -176> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: _check_auth_tickets 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr: -175> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: handle_config config(0 keys) 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr: -174> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr: -173> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr: -172> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: dump: 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.717 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -171> 2024-05-09T20:27:21.592+0000 7f1879ffb640 4 set_mon_vals no callback set 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -170> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 monclient: authenticate success, global_id 6801 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -169> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -168> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -167> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -166> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -165> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: init done 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -164> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cr dump hook 0x558458a92f08 2024-05-09T20:27:21.718 INFO:tasks.workunit.client.0.smithi063.stderr: -163> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -162> 2024-05-09T20:27:21.596+0000 7f1867fff640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -161> 2024-05-09T20:27:21.596+0000 7f1867fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -160> 2024-05-09T20:27:21.596+0000 7f1867fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -159> 2024-05-09T20:27:21.596+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f1860066d60 auth_method 0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -158> 2024-05-09T20:27:21.596+0000 7f19962d9640 10 monclient: get_auth_request con 0x558458ab2030 auth_method 0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -157> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -156> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 realm 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -155> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -154> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -153> 2024-05-09T20:27:21.596+0000 7f19987c5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -152> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -151> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -150> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -149> 2024-05-09T20:27:21.596+0000 7f1995ad8640 10 monclient: get_auth_request con 0x558458ab6270 auth_method 0 2024-05-09T20:27:21.719 INFO:tasks.workunit.client.0.smithi063.stderr: -148> 2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -147> 2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -146> 2024-05-09T20:27:21.600+0000 7f1996ada640 10 monclient: get_auth_request con 0x558458ab9ad0 auth_method 0 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -145> 2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -144> 2024-05-09T20:27:21.600+0000 7f19987c5980 20 searching for the correct realm 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -143> 2024-05-09T20:27:21.600+0000 7f19962d9640 10 monclient: get_auth_request con 0x558458abe850 auth_method 0 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -142> 2024-05-09T20:27:21.600+0000 7f1995ad8640 10 monclient: get_auth_request con 0x7f1988008030 auth_method 0 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -141> 2024-05-09T20:27:21.600+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f198c0a58f0 auth_method 0 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -140> 2024-05-09T20:27:21.604+0000 7f1996ada640 0 -- 172.21.15.63:0/1395419920 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f198c0a58f0 msgr2=0x7f198c110890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -139> 2024-05-09T20:27:21.604+0000 7f1867fff640 1 client.6801.objecter ms_handle_reset 0x7f198c0a58f0 session 0x7f198c0a6070 osd.2 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -138> 2024-05-09T20:27:21.604+0000 7f1867fff640 10 monclient: _renew_subs 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -137> 2024-05-09T20:27:21.604+0000 7f1867fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -136> 2024-05-09T20:27:21.604+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f18600774f0 auth_method 0 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -135> 2024-05-09T20:27:21.604+0000 7f19962d9640 10 monclient: get_auth_request con 0x7f1990033070 auth_method 0 2024-05-09T20:27:21.720 INFO:tasks.workunit.client.0.smithi063.stderr: -134> 2024-05-09T20:27:21.608+0000 7f1995ad8640 10 monclient: get_auth_request con 0x7f1988020c00 auth_method 0 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -133> 2024-05-09T20:27:21.608+0000 7f1995ad8640 0 -- 172.21.15.63:0/1395419920 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f1988008030 msgr2=0x7f198802c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -132> 2024-05-09T20:27:21.608+0000 7f1867fff640 1 client.6801.objecter ms_handle_reset 0x7f1988008030 session 0x7f1988040040 osd.7 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -131> 2024-05-09T20:27:21.608+0000 7f1995ad8640 10 monclient: get_auth_request con 0x7f198c0a58f0 auth_method 0 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -130> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -129> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -128> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -127> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -126> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -125> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -124> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -123> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -122> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -121> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -120> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.721 INFO:tasks.workunit.client.0.smithi063.stderr: -119> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -118> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 zone default found 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -117> 2024-05-09T20:27:21.620+0000 7f19987c5980 4 Realm: () 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -116> 2024-05-09T20:27:21.620+0000 7f19987c5980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -115> 2024-05-09T20:27:21.620+0000 7f19987c5980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -114> 2024-05-09T20:27:21.620+0000 7f19987c5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -113> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 zonegroup default 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -112> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -111> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -110> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -109> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -108> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -107> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -106> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 started sync module instance, tier type = 2024-05-09T20:27:21.722 INFO:tasks.workunit.client.0.smithi063.stderr: -105> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -104> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=0 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -103> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=1 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -102> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=2 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -101> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=3 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -100> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=4 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -99> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=5 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -98> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=6 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -97> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 add_watcher() i=7 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -96> 2024-05-09T20:27:21.640+0000 7f19987c5980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -95> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding auth protocol: cephx 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -94> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding auth protocol: cephx 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -93> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding auth protocol: cephx 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -92> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -91> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.723 INFO:tasks.workunit.client.0.smithi063.stderr: -90> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -89> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -88> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -87> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -86> 2024-05-09T20:27:21.640+0000 7f184d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -85> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -84> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -83> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -82> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -81> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -80> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -79> 2024-05-09T20:27:21.640+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -78> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -77> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:21.724 INFO:tasks.workunit.client.0.smithi063.stderr: -76> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cache list hook 0x558458a8a410 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -75> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cache inspect hook 0x558458a8a410 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -74> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cache erase hook 0x558458a8a410 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -73> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cache zap hook 0x558458a8a410 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -72> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 note: GC not initialized 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -71> 2024-05-09T20:27:21.640+0000 7f18477fe640 20 reqs_thread_entry: start 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -70> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command sync trace show hook 0x7f198c0043b0 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -69> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command sync trace history hook 0x7f198c0043b0 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -68> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command sync trace active hook 0x7f198c0043b0 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -67> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command sync trace active_short hook 0x7f198c0043b0 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -66> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -65> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 Filter name: none 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -64> 2024-05-09T20:27:21.640+0000 7f18457fa640 20 reqs_thread_entry: start 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -63> 2024-05-09T20:27:21.640+0000 7f19987c5980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -62> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.725 INFO:tasks.workunit.client.0.smithi063.stderr: -61> 2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -60> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -59> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -58> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 : miss 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -57> 2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -56> 2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -55> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -54> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -53> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -52> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -51> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -50> 2024-05-09T20:27:21.652+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -49> 2024-05-09T20:27:21.652+0000 7f19987c5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -48> 2024-05-09T20:27:21.652+0000 7f19987c5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.726 INFO:tasks.workunit.client.0.smithi063.stderr: -47> 2024-05-09T20:27:21.652+0000 7f19987c5980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26, ofs0, ns] 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -46> 2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431598 cookie 94026911740896 notifier 6801 bl.length()=923 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -45> 2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -44> 2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -43> 2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -42> 2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_sources= new_sources= 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -41> 2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential sources added=[] removed=[] 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -40> 2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_dests= new_dests= 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -39> 2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential dests added=[] removed=[] 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -38> 2024-05-09T20:27:21.660+0000 7f1867fff640 1 client.6801.objecter ms_handle_reset 0x558458ab9ad0 session 0x558458ab5600 osd.0 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -37> 2024-05-09T20:27:21.660+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f1988008030 auth_method 0 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -36> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.0 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -35> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.0 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -34> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.1 2024-05-09T20:27:21.727 INFO:tasks.workunit.client.0.smithi063.stderr: -33> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.1 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -32> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.2 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -31> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.2 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -30> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.3 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -29> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.3 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -28> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.4 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -27> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.4 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -26> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.5 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -25> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.5 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -24> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.6 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -23> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.6 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -22> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.7 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -21> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.7 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -20> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.8 2024-05-09T20:27:21.728 INFO:tasks.workunit.client.0.smithi063.stderr: -19> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.8 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -18> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.9 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -17> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.9 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -16> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.10 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -15> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.10 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -14> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.11 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -13> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.11 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -12> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.0 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -11> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -10> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.2 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -9> 2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.3 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -8> 2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.4 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -7> 2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.5 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -6> 2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.6 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -5> 2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.7 2024-05-09T20:27:21.729 INFO:tasks.workunit.client.0.smithi063.stderr: -4> 2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.8 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: -3> 2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.9 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: -2> 2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.10 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: -1> 2024-05-09T20:27:21.676+0000 7f19987c5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f19987c5980 time 2024-05-09T20:27:21.677617+0000 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f199c09be25] 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f199c09bfd9] 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55845744eca1] 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x57f) [0x55845745f67f] 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f199af05d90] 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:27:21.730 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 0> 2024-05-09T20:27:21.676+0000 7f19987c5980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f19987c5980 thread_name:radosgw-admin 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f199af1e520] 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f199c09be7f] 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f199c09bfd9] 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55845744eca1] 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x57f) [0x55845745f67f] 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:27:21.731 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f199af05d90] 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr:--- logging levels --- 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 none 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 lockdep 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 context 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 crush 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_balancer 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_locker 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log 2024-05-09T20:27:21.732 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log_expire 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_migrator 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 3/ 5 mds_quiesce 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 buffer 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 timer 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 filer 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 striper 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 objecter 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rados 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_mirror 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_replay 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_pwl 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 journaler 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objectcacher 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 immutable_obj_cache 2024-05-09T20:27:21.733 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 client 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 osd 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 optracker 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objclass 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 filestore 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 journal 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 ms 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mon 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 0/10 monc 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 paxos 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 tp 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 auth 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 crypto 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 finisher 2024-05-09T20:27:21.734 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 reserver 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 heartbeatmap 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 perfcounter 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 20/20 rgw 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_sync 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_datacache 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_access 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_dbstore 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_flight 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_lifecycle 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_notification 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 javaclient 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 asok 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 throttle 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 refs 2024-05-09T20:27:21.735 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 compressor 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluestore 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluefs 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 bdev 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 kstore 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 4/ 5 rocksdb 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 fuse 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 2/ 5 mgr 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mgrc 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 dpdk 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 eventtrace 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 prioritycache 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 test 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephfs_mirror 2024-05-09T20:27:21.736 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephsqlite 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 crimson_interrupt 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_onode 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_odata 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_omap 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_tm 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_t 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cleaner 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_epm 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_lba 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cache 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_journal 2024-05-09T20:27:21.737 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_device 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_backref 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 alienstore 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mclock 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cyanstore 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 ceph_exporter 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 memstore 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 trace 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 ceph_dedup 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: -2/-2 (syslog threshold) 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 99/99 (stderr threshold) 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 7f18457fa640 / http_manager 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 7f18477fe640 / http_manager 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 7f184d7e2640 / rgw_dt_lg_renew 2024-05-09T20:27:21.738 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1867fff640 / ms_dispatch 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1879ffb640 / io_context_pool 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f187bfff640 / ms_dispatch 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1882e05640 / ms_dispatch 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1884608640 / io_context_pool 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1995ad8640 / msgr-worker-2 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f19962d9640 / msgr-worker-1 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1996ada640 / msgr-worker-0 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f19972db640 / io_context_pool 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: 7f19987c5980 / radosgw-admin 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: max_recent 500 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: max_new 1000 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr: log_file /var/log/ceph/ceph-client.admin.94497.log 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr:--- end dump of recent events --- 2024-05-09T20:27:21.739 INFO:tasks.workunit.client.0.smithi063.stderr:--- begin dump of recent events --- 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -499> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding auth protocol: cephx 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -498> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding auth protocol: cephx 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -497> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding auth protocol: cephx 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -496> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -495> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -494> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -493> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -492> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -491> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -490> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -489> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -488> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -487> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -486> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: crc 2024-05-09T20:27:21.740 INFO:tasks.workunit.client.0.smithi063.stderr: -485> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 AuthRegistry(0x5584589c59c0) adding con mode: secure 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -484> 2024-05-09T20:27:21.576+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -483> 2024-05-09T20:27:21.576+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -482> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 asok(0x5584587bc430) register_command rotate-key hook 0x5584589c5b08 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -481> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _reopen_session rank -1 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -480> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _add_conns ranks=[0,2,1] 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -479> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-a con 0x558458877360 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -478> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-c con 0x558458881d70 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -477> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-b con 0x55845890c970 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -476> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -475> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -474> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -473> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): _renew_subs 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -472> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient(hunting): authenticate will time out at 2048.717521s 2024-05-09T20:27:21.741 INFO:tasks.workunit.client.0.smithi063.stderr: -471> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): get_auth_request con 0x558458877360 auth_method 0 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -470> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -469> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -468> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -467> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): get_auth_request con 0x558458881d70 auth_method 0 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -466> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -465> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -464> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -463> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -462> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -461> 2024-05-09T20:27:21.576+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -460> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -459> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -458> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -457> 2024-05-09T20:27:21.576+0000 7f1996ada640 10 monclient(hunting): get_auth_request con 0x55845890c970 auth_method 0 2024-05-09T20:27:21.742 INFO:tasks.workunit.client.0.smithi063.stderr: -456> 2024-05-09T20:27:21.576+0000 7f1996ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -455> 2024-05-09T20:27:21.576+0000 7f1996ada640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -454> 2024-05-09T20:27:21.576+0000 7f1996ada640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -453> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient(hunting): handle_auth_done global_id 6407 payload 824 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -452> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient: _finish_hunting 0 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -451> 2024-05-09T20:27:21.576+0000 7f1995ad8640 1 monclient: found mon.noname-c 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -450> 2024-05-09T20:27:21.576+0000 7f1995ad8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -449> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -448> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr: -447> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: dump: 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.743 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr: -446> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: _finish_auth 0 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr: -445> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: _check_auth_tickets 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr: -444> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: handle_config config(0 keys) 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr: -443> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr: -442> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr: -441> 2024-05-09T20:27:21.576+0000 7f1882e05640 10 monclient: dump: 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.744 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -440> 2024-05-09T20:27:21.576+0000 7f1884608640 4 set_mon_vals no callback set 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -439> 2024-05-09T20:27:21.576+0000 7f19987c5980 5 monclient: authenticate success, global_id 6407 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -438> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -437> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -436> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -435> 2024-05-09T20:27:21.576+0000 7f19987c5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -434> 2024-05-09T20:27:21.576+0000 7f19987c5980 1 librados: init done 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -433> 2024-05-09T20:27:21.580+0000 7f1882e05640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:27:21.745 INFO:tasks.workunit.client.0.smithi063.stderr: -432> 2024-05-09T20:27:21.580+0000 7f1882e05640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -431> 2024-05-09T20:27:21.580+0000 7f1882e05640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -430> 2024-05-09T20:27:21.580+0000 7f19962d9640 10 monclient: get_auth_request con 0x7f187c066d60 auth_method 0 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -429> 2024-05-09T20:27:21.580+0000 7f1996ada640 10 monclient: get_auth_request con 0x5584589ed1e0 auth_method 0 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -428> 2024-05-09T20:27:21.580+0000 7f19962d9640 10 monclient: get_auth_request con 0x558458a10640 auth_method 0 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -427> 2024-05-09T20:27:21.580+0000 7f1995ad8640 10 monclient: get_auth_request con 0x558458a149a0 auth_method 0 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -426> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient: get_auth_request con 0x558458a13ef0 auth_method 0 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -425> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: build_initial_monmap 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -424> 2024-05-09T20:27:21.584+0000 7f19987c5980 1 build_initial for_mkfs: 0 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr: -423> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: monmap: 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:27:21.746 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -422> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding auth protocol: cephx 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -421> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding auth protocol: cephx 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -420> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding auth protocol: cephx 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -419> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -418> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -417> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -416> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -415> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -414> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.747 INFO:tasks.workunit.client.0.smithi063.stderr: -413> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -412> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -411> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -410> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -409> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: crc 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -408> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1ca90) adding con mode: secure 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -407> 2024-05-09T20:27:21.584+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -406> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 asok(0x5584587bc430) register_command objecter_requests cmddesc objecter_requests hook 0x558458a22000 EEXIST 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -405> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: init 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -404> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding auth protocol: cephx 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -403> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding auth protocol: cephx 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -402> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding auth protocol: cephx 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -401> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -400> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.748 INFO:tasks.workunit.client.0.smithi063.stderr: -399> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -398> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -397> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -396> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -395> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -394> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -393> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -392> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -391> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: crc 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -390> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 AuthRegistry(0x558458a1b918) adding con mode: secure 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -389> 2024-05-09T20:27:21.584+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -388> 2024-05-09T20:27:21.584+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -387> 2024-05-09T20:27:21.584+0000 7f19987c5980 5 asok(0x5584587bc430) register_command rotate-key cmddesc rotate-key hook 0x558458a1ba60 EEXIST 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -386> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: _reopen_session rank -1 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -385> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-09T20:27:21.749 INFO:tasks.workunit.client.0.smithi063.stderr: -384> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-c con 0x558458a46c60 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -383> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-a con 0x558458a47040 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -382> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-b con 0x558458a42200 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -381> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -380> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -379> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -378> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): _renew_subs 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -377> 2024-05-09T20:27:21.584+0000 7f19987c5980 10 monclient(hunting): authenticate will time out at 2048.727502s 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -376> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): get_auth_request con 0x558458a47040 auth_method 0 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -375> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -374> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -373> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -372> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): get_auth_request con 0x558458a46c60 auth_method 0 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -371> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.750 INFO:tasks.workunit.client.0.smithi063.stderr: -370> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -369> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -368> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -367> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -366> 2024-05-09T20:27:21.584+0000 7f1996ada640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -365> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -364> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -363> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -362> 2024-05-09T20:27:21.584+0000 7f19962d9640 10 monclient(hunting): get_auth_request con 0x558458a42200 auth_method 0 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -361> 2024-05-09T20:27:21.584+0000 7f19962d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -360> 2024-05-09T20:27:21.584+0000 7f19962d9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -359> 2024-05-09T20:27:21.584+0000 7f19962d9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -358> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient(hunting): handle_auth_done global_id 6410 payload 824 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -357> 2024-05-09T20:27:21.584+0000 7f1995ad8640 10 monclient: _finish_hunting 0 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -356> 2024-05-09T20:27:21.584+0000 7f1995ad8640 1 monclient: found mon.noname-c 2024-05-09T20:27:21.751 INFO:tasks.workunit.client.0.smithi063.stderr: -355> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr: -354> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr: -353> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr: -352> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: dump: 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.752 INFO:tasks.workunit.client.0.smithi063.stderr: -351> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: _finish_auth 0 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr: -350> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: _check_auth_tickets 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr: -349> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: handle_config config(0 keys) 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr: -348> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr: -347> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr: -346> 2024-05-09T20:27:21.588+0000 7f187bfff640 10 monclient: dump: 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.753 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -345> 2024-05-09T20:27:21.588+0000 7f19972db640 4 set_mon_vals no callback set 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -344> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 monclient: authenticate success, global_id 6410 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -343> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -342> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -341> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -340> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -339> 2024-05-09T20:27:21.588+0000 7f187bfff640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -338> 2024-05-09T20:27:21.588+0000 7f187bfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -337> 2024-05-09T20:27:21.588+0000 7f187bfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -336> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f1870066ca0 auth_method 0 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -335> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: get_monmap_and_config 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -334> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: build_initial_monmap 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -333> 2024-05-09T20:27:21.588+0000 7f19987c5980 1 build_initial for_mkfs: 0 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr: -332> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: monmap: 2024-05-09T20:27:21.754 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr: -331> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr: -330> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr: -329> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr: -328> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr: -327> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.755 INFO:tasks.workunit.client.0.smithi063.stderr: -326> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -325> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -324> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -323> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -322> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -321> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -320> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -319> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -318> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -317> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -316> 2024-05-09T20:27:21.588+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -315> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: init 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -314> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding auth protocol: cephx 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -313> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding auth protocol: cephx 2024-05-09T20:27:21.756 INFO:tasks.workunit.client.0.smithi063.stderr: -312> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding auth protocol: cephx 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -311> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -310> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -309> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -308> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -307> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -306> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -305> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -304> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -303> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -302> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -301> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: crc 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -300> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b25bf0) adding con mode: secure 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -299> 2024-05-09T20:27:21.588+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -298> 2024-05-09T20:27:21.588+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.757 INFO:tasks.workunit.client.0.smithi063.stderr: -297> 2024-05-09T20:27:21.588+0000 7f19987c5980 5 asok(0x5584587bc430) register_command rotate-key cmddesc rotate-key hook 0x7ffdf0b25d38 EEXIST 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -296> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _reopen_session rank -1 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -295> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient: _add_conns ranks=[2,1,0] 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -294> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-c con 0x558458a98da0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -293> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-b con 0x558458a942b0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -292> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-a con 0x558458a9de90 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -291> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -290> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -289> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -288> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): _renew_subs 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -287> 2024-05-09T20:27:21.588+0000 7f19987c5980 10 monclient(hunting): authenticate will time out at 2048.731501s 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -286> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): get_auth_request con 0x558458a98da0 auth_method 0 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -285> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -284> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.758 INFO:tasks.workunit.client.0.smithi063.stderr: -283> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -282> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): get_auth_request con 0x558458a9de90 auth_method 0 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -281> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -280> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -279> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -278> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -277> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -276> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -275> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -274> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -273> 2024-05-09T20:27:21.588+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -272> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient(hunting): get_auth_request con 0x558458a942b0 auth_method 0 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -271> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -270> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.759 INFO:tasks.workunit.client.0.smithi063.stderr: -269> 2024-05-09T20:27:21.588+0000 7f1996ada640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr: -268> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient(hunting): handle_auth_done global_id 6413 payload 274 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr: -267> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient: _finish_hunting 0 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr: -266> 2024-05-09T20:27:21.588+0000 7f1995ad8640 1 monclient: found mon.noname-c 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr: -265> 2024-05-09T20:27:21.588+0000 7f1995ad8640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr: -264> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr: -263> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr: -262> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: dump: 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.760 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -261> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: _finish_auth 0 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -260> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: _check_auth_tickets 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -259> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 monclient: authenticate success, global_id 6413 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -258> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: handle_config config(0 keys) 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -257> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: get_monmap_and_config success 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -256> 2024-05-09T20:27:21.592+0000 7f19987c5980 4 set_mon_vals no callback set 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -255> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -254> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr: -253> 2024-05-09T20:27:21.592+0000 7f1879ffb640 10 monclient: dump: 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.761 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr: -252> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: shutdown 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr: -251> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: build_initial_monmap 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr: -250> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 build_initial for_mkfs: 0 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr: -249> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: monmap: 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:27:21.762 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: -248> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: -247> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: -246> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding auth protocol: cephx 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: -245> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: -244> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: -243> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: -242> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.763 INFO:tasks.workunit.client.0.smithi063.stderr: -241> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -240> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -239> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -238> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -237> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -236> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -235> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: crc 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -234> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a647b0) adding con mode: secure 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -233> 2024-05-09T20:27:21.592+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -232> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: starting msgr at 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -231> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: starting objecter 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -230> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 asok(0x5584587bc430) register_command objecter_requests cmddesc objecter_requests hook 0x558458a941c0 EEXIST 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -229> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: setting wanted keys 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -228> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: calling monclient init 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -227> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: init 2024-05-09T20:27:21.764 INFO:tasks.workunit.client.0.smithi063.stderr: -226> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding auth protocol: cephx 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -225> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding auth protocol: cephx 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -224> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding auth protocol: cephx 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -223> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -222> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -221> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -220> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -219> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -218> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -217> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -216> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -215> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -214> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -213> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: crc 2024-05-09T20:27:21.765 INFO:tasks.workunit.client.0.smithi063.stderr: -212> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 AuthRegistry(0x558458a63980) adding con mode: secure 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -211> 2024-05-09T20:27:21.592+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -210> 2024-05-09T20:27:21.592+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -209> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 asok(0x5584587bc430) register_command rotate-key cmddesc rotate-key hook 0x558458a63ac8 EEXIST 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -208> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _reopen_session rank -1 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -207> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -206> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-a con 0x558458a942b0 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -205> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-b con 0x558458a98da0 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -204> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): picked mon.noname-c con 0x558458a9de90 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -203> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -202> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -201> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): start opening mon connection 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -200> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): _renew_subs 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -199> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient(hunting): authenticate will time out at 2048.734079s 2024-05-09T20:27:21.766 INFO:tasks.workunit.client.0.smithi063.stderr: -198> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): get_auth_request con 0x558458a942b0 auth_method 0 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -197> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -196> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -195> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -194> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): get_auth_request con 0x558458a9de90 auth_method 0 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -193> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -192> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -191> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -190> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -189> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -188> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -187> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -186> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -185> 2024-05-09T20:27:21.592+0000 7f19962d9640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:27:21.767 INFO:tasks.workunit.client.0.smithi063.stderr: -184> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient(hunting): handle_auth_done global_id 6801 payload 824 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr: -183> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient: _finish_hunting 0 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr: -182> 2024-05-09T20:27:21.592+0000 7f1995ad8640 1 monclient: found mon.noname-a 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr: -181> 2024-05-09T20:27:21.592+0000 7f1995ad8640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr: -180> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr: -179> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr: -178> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: dump: 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.768 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr: -177> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: _finish_auth 0 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr: -176> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: _check_auth_tickets 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr: -175> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: handle_config config(0 keys) 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr: -174> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr: -173> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr: -172> 2024-05-09T20:27:21.592+0000 7f1867fff640 10 monclient: dump: 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:27:21.769 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -171> 2024-05-09T20:27:21.592+0000 7f1879ffb640 4 set_mon_vals no callback set 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -170> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 monclient: authenticate success, global_id 6801 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -169> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -168> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -167> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _renew_subs 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -166> 2024-05-09T20:27:21.592+0000 7f19987c5980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -165> 2024-05-09T20:27:21.592+0000 7f19987c5980 1 librados: init done 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -164> 2024-05-09T20:27:21.592+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cr dump hook 0x558458a92f08 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -163> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.770 INFO:tasks.workunit.client.0.smithi063.stderr: -162> 2024-05-09T20:27:21.596+0000 7f1867fff640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -161> 2024-05-09T20:27:21.596+0000 7f1867fff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -160> 2024-05-09T20:27:21.596+0000 7f1867fff640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -159> 2024-05-09T20:27:21.596+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f1860066d60 auth_method 0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -158> 2024-05-09T20:27:21.596+0000 7f19962d9640 10 monclient: get_auth_request con 0x558458ab2030 auth_method 0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -157> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -156> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 realm 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -155> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -154> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -153> 2024-05-09T20:27:21.596+0000 7f19987c5980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -152> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -151> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -150> 2024-05-09T20:27:21.596+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -149> 2024-05-09T20:27:21.596+0000 7f1995ad8640 10 monclient: get_auth_request con 0x558458ab6270 auth_method 0 2024-05-09T20:27:21.771 INFO:tasks.workunit.client.0.smithi063.stderr: -148> 2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -147> 2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -146> 2024-05-09T20:27:21.600+0000 7f1996ada640 10 monclient: get_auth_request con 0x558458ab9ad0 auth_method 0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -145> 2024-05-09T20:27:21.600+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -144> 2024-05-09T20:27:21.600+0000 7f19987c5980 20 searching for the correct realm 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -143> 2024-05-09T20:27:21.600+0000 7f19962d9640 10 monclient: get_auth_request con 0x558458abe850 auth_method 0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -142> 2024-05-09T20:27:21.600+0000 7f1995ad8640 10 monclient: get_auth_request con 0x7f1988008030 auth_method 0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -141> 2024-05-09T20:27:21.600+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f198c0a58f0 auth_method 0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -140> 2024-05-09T20:27:21.604+0000 7f1996ada640 0 -- 172.21.15.63:0/1395419920 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f198c0a58f0 msgr2=0x7f198c110890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -139> 2024-05-09T20:27:21.604+0000 7f1867fff640 1 client.6801.objecter ms_handle_reset 0x7f198c0a58f0 session 0x7f198c0a6070 osd.2 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -138> 2024-05-09T20:27:21.604+0000 7f1867fff640 10 monclient: _renew_subs 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -137> 2024-05-09T20:27:21.604+0000 7f1867fff640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -136> 2024-05-09T20:27:21.604+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f18600774f0 auth_method 0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -135> 2024-05-09T20:27:21.604+0000 7f19962d9640 10 monclient: get_auth_request con 0x7f1990033070 auth_method 0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -134> 2024-05-09T20:27:21.608+0000 7f1995ad8640 10 monclient: get_auth_request con 0x7f1988020c00 auth_method 0 2024-05-09T20:27:21.772 INFO:tasks.workunit.client.0.smithi063.stderr: -133> 2024-05-09T20:27:21.608+0000 7f1995ad8640 0 -- 172.21.15.63:0/1395419920 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f1988008030 msgr2=0x7f198802c950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -132> 2024-05-09T20:27:21.608+0000 7f1867fff640 1 client.6801.objecter ms_handle_reset 0x7f1988008030 session 0x7f1988040040 osd.7 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -131> 2024-05-09T20:27:21.608+0000 7f1995ad8640 10 monclient: get_auth_request con 0x7f198c0a58f0 auth_method 0 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -130> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -129> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -128> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -127> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -126> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -125> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -124> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -123> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -122> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -121> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -120> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.773 INFO:tasks.workunit.client.0.smithi063.stderr: -119> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -118> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 zone default found 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -117> 2024-05-09T20:27:21.620+0000 7f19987c5980 4 Realm: () 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -116> 2024-05-09T20:27:21.620+0000 7f19987c5980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -115> 2024-05-09T20:27:21.620+0000 7f19987c5980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -114> 2024-05-09T20:27:21.620+0000 7f19987c5980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -113> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 zonegroup default 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -112> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -111> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -110> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -109> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -108> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -107> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -106> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 started sync module instance, tier type = 2024-05-09T20:27:21.774 INFO:tasks.workunit.client.0.smithi063.stderr: -105> 2024-05-09T20:27:21.620+0000 7f19987c5980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -104> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=0 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -103> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=1 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -102> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=2 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -101> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=3 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -100> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=4 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -99> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=5 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -98> 2024-05-09T20:27:21.636+0000 7f19987c5980 20 add_watcher() i=6 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -97> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 add_watcher() i=7 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -96> 2024-05-09T20:27:21.640+0000 7f19987c5980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -95> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding auth protocol: cephx 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -94> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding auth protocol: cephx 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -93> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding auth protocol: cephx 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -92> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -91> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.775 INFO:tasks.workunit.client.0.smithi063.stderr: -90> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -89> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -88> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -87> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -86> 2024-05-09T20:27:21.640+0000 7f184d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -85> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -84> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -83> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -82> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -81> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: crc 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -80> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 AuthRegistry(0x7ffdf0b26080) adding con mode: secure 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -79> 2024-05-09T20:27:21.640+0000 7f19987c5980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -78> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -77> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:21.776 INFO:tasks.workunit.client.0.smithi063.stderr: -76> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cache list hook 0x558458a8a410 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -75> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cache inspect hook 0x558458a8a410 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -74> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cache erase hook 0x558458a8a410 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -73> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command cache zap hook 0x558458a8a410 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -72> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 note: GC not initialized 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -71> 2024-05-09T20:27:21.640+0000 7f18477fe640 20 reqs_thread_entry: start 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -70> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command sync trace show hook 0x7f198c0043b0 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -69> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command sync trace history hook 0x7f198c0043b0 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -68> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command sync trace active hook 0x7f198c0043b0 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -67> 2024-05-09T20:27:21.640+0000 7f19987c5980 5 asok(0x5584587bc430) register_command sync trace active_short hook 0x7f198c0043b0 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -66> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -65> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 Filter name: none 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -64> 2024-05-09T20:27:21.640+0000 7f18457fa640 20 reqs_thread_entry: start 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -63> 2024-05-09T20:27:21.640+0000 7f19987c5980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -62> 2024-05-09T20:27:21.640+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.777 INFO:tasks.workunit.client.0.smithi063.stderr: -61> 2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -60> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -59> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -58> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 : miss 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -57> 2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -56> 2024-05-09T20:27:21.644+0000 7f19987c5980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -55> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -54> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -53> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -52> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -51> 2024-05-09T20:27:21.644+0000 7f19987c5980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -50> 2024-05-09T20:27:21.652+0000 7f19987c5980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -49> 2024-05-09T20:27:21.652+0000 7f19987c5980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.778 INFO:tasks.workunit.client.0.smithi063.stderr: -48> 2024-05-09T20:27:21.652+0000 7f19987c5980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -47> 2024-05-09T20:27:21.652+0000 7f19987c5980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26, ofs0, ns] 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -46> 2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431598 cookie 94026911740896 notifier 6801 bl.length()=923 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -45> 2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -44> 2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -43> 2024-05-09T20:27:21.652+0000 7f1879ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -42> 2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_sources= new_sources= 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -41> 2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential sources added=[] removed=[] 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -40> 2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_dests= new_dests= 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -39> 2024-05-09T20:27:21.656+0000 7f19987c5980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential dests added=[] removed=[] 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -38> 2024-05-09T20:27:21.660+0000 7f1867fff640 1 client.6801.objecter ms_handle_reset 0x558458ab9ad0 session 0x558458ab5600 osd.0 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -37> 2024-05-09T20:27:21.660+0000 7f1996ada640 10 monclient: get_auth_request con 0x7f1988008030 auth_method 0 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -36> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.0 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -35> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.0 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -34> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.1 2024-05-09T20:27:21.779 INFO:tasks.workunit.client.0.smithi063.stderr: -33> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.1 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -32> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.2 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -31> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.2 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -30> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.3 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -29> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.3 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -28> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.4 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -27> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.4 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -26> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.5 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -25> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.5 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -24> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.6 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -23> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.6 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -22> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.7 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -21> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.7 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -20> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.8 2024-05-09T20:27:21.780 INFO:tasks.workunit.client.0.smithi063.stderr: -19> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.8 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -18> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.9 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -17> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.9 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -16> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.10 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -15> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.10 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -14> 2024-05-09T20:27:21.660+0000 7f19987c5980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.11 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -13> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.11 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -12> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.0 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -11> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -10> 2024-05-09T20:27:21.660+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.2 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -9> 2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.3 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -8> 2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.4 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -7> 2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.5 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -6> 2024-05-09T20:27:21.664+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.6 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -5> 2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.7 2024-05-09T20:27:21.781 INFO:tasks.workunit.client.0.smithi063.stderr: -4> 2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.8 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: -3> 2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.9 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: -2> 2024-05-09T20:27:21.668+0000 7f19987c5980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.10 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: -1> 2024-05-09T20:27:21.676+0000 7f19987c5980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f19987c5980 time 2024-05-09T20:27:21.677617+0000 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f199c09be25] 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f199c09bfd9] 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55845744eca1] 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x57f) [0x55845745f67f] 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f199af05d90] 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:27:21.782 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 0> 2024-05-09T20:27:21.676+0000 7f19987c5980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f19987c5980 thread_name:radosgw-admin 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f199af1e520] 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f199c09be7f] 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f199c09bfd9] 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55845744eca1] 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x57f) [0x55845745f67f] 2024-05-09T20:27:21.783 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f199af05d90] 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr:--- logging levels --- 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 none 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 lockdep 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 context 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 crush 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_balancer 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_locker 2024-05-09T20:27:21.784 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log_expire 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_migrator 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 3/ 5 mds_quiesce 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 buffer 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 timer 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 filer 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 striper 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 objecter 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rados 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_mirror 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_replay 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_pwl 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 journaler 2024-05-09T20:27:21.785 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objectcacher 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 immutable_obj_cache 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 client 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 osd 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 optracker 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objclass 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 filestore 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 journal 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 ms 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mon 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 0/10 monc 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 paxos 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 tp 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 auth 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 crypto 2024-05-09T20:27:21.786 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 finisher 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 reserver 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 heartbeatmap 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 perfcounter 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 20/20 rgw 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_sync 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_datacache 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_access 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_dbstore 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_flight 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_lifecycle 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_notification 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 javaclient 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 asok 2024-05-09T20:27:21.787 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 throttle 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 refs 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 compressor 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluestore 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluefs 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 bdev 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 kstore 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 4/ 5 rocksdb 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 fuse 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 2/ 5 mgr 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mgrc 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 dpdk 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 eventtrace 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 prioritycache 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 test 2024-05-09T20:27:21.788 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephfs_mirror 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephsqlite 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 crimson_interrupt 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_onode 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_odata 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_omap 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_tm 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_t 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cleaner 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_epm 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_lba 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cache 2024-05-09T20:27:21.789 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_journal 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_device 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_backref 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 alienstore 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mclock 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cyanstore 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 ceph_exporter 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 memstore 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 trace 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 ceph_dedup 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: -2/-2 (syslog threshold) 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 99/99 (stderr threshold) 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 7f18457fa640 / http_manager 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 7f18477fe640 / http_manager 2024-05-09T20:27:21.790 INFO:tasks.workunit.client.0.smithi063.stderr: 7f184d7e2640 / rgw_dt_lg_renew 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1867fff640 / ms_dispatch 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1879ffb640 / io_context_pool 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f187bfff640 / ms_dispatch 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1882e05640 / ms_dispatch 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1884608640 / io_context_pool 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1995ad8640 / msgr-worker-2 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f19962d9640 / msgr-worker-1 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f1996ada640 / msgr-worker-0 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f19972db640 / io_context_pool 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: 7f19987c5980 / radosgw-admin 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: max_recent 500 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: max_new 1000 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr: log_file /var/lib/ceph/crash/2024-05-09T20:27:21.679593Z_419612aa-6f5e-42e8-a6ab-8d8872aa95c9/log 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr:--- end dump of recent events --- 2024-05-09T20:27:21.791 INFO:tasks.workunit.client.0.smithi063.stderr:Aborted 2024-05-09T20:27:21.792 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-05-09T20:27:21.793 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:21.793 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:21.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 20 realm 2024-05-09T20:27:21.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:21.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.756+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.760+0000 7f955d6ca980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.760+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.760+0000 7f955d6ca980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:21.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.760+0000 7f955d6ca980 20 searching for the correct realm 2024-05-09T20:27:21.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.764+0000 7f955b1de640 0 -- 172.21.15.63:0/2606840159 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f9554038070 msgr2=0x7f955402b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:21.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.980+0000 7f955b1de640 0 -- 172.21.15.63:0/2606840159 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f9554038070 msgr2=0x7f955402b910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:21.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:21.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:21.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:21.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:21.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:21.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:21.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:21.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.992+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 zone default found 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 4 Realm: () 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 zonegroup default 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.987 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:21.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:21.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 started sync module instance, tier type = 2024-05-09T20:27:21.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:21.996+0000 7f955d6ca980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:22.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.020+0000 7f955d6ca980 20 add_watcher() i=0 2024-05-09T20:27:22.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.020+0000 7f955d6ca980 20 add_watcher() i=1 2024-05-09T20:27:22.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.020+0000 7f955d6ca980 20 add_watcher() i=2 2024-05-09T20:27:22.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.020+0000 7f955d6ca980 20 add_watcher() i=3 2024-05-09T20:27:22.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.020+0000 7f955d6ca980 20 add_watcher() i=4 2024-05-09T20:27:22.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.020+0000 7f955d6ca980 20 add_watcher() i=5 2024-05-09T20:27:22.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.020+0000 7f955d6ca980 20 add_watcher() i=6 2024-05-09T20:27:22.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.024+0000 7f955d6ca980 20 add_watcher() i=7 2024-05-09T20:27:22.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.024+0000 7f955d6ca980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:22.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.028+0000 7f9411fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:22.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.028+0000 7f955d6ca980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:22.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.028+0000 7f955d6ca980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:22.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.028+0000 7f955d6ca980 5 note: GC not initialized 2024-05-09T20:27:22.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.028+0000 7f9410fe1640 20 reqs_thread_entry: start 2024-05-09T20:27:22.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:22.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 Filter name: none 2024-05-09T20:27:22.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f938a7fc640 20 reqs_thread_entry: start 2024-05-09T20:27:22.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:22.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:22.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:22.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 rados_obj.operate() r=0 bl.length=443 2024-05-09T20:27:22.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:22.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:22.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:22.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 rados->read ofs=0 len=0 2024-05-09T20:27:22.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 rados_obj.operate() r=0 bl.length=443 2024-05-09T20:27:22.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.032+0000 7f955d6ca980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:22.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.036+0000 7f9411fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:22.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.040+0000 7f955d6ca980 20 remove_watcher() i=0 2024-05-09T20:27:22.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.040+0000 7f955d6ca980 2 removed watcher, disabling cache 2024-05-09T20:27:22.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.044+0000 7f955d6ca980 20 remove_watcher() i=1 2024-05-09T20:27:22.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.048+0000 7f955d6ca980 20 remove_watcher() i=2 2024-05-09T20:27:22.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.048+0000 7f955d6ca980 20 remove_watcher() i=3 2024-05-09T20:27:22.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.052+0000 7f955d6ca980 20 remove_watcher() i=4 2024-05-09T20:27:22.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.052+0000 7f955d6ca980 20 remove_watcher() i=5 2024-05-09T20:27:22.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.052+0000 7f955d6ca980 20 remove_watcher() i=6 2024-05-09T20:27:22.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:22.056+0000 7f955d6ca980 20 remove_watcher() i=7 2024-05-09T20:27:22.068 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:22.069 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:22.069 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-commit-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:27:22.069 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:52.464 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-commit-target-layout --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:27:52.500 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:52.500 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:52.548 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.556+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.556+0000 7f38e05ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.556+0000 7f38e05ed980 20 realm 2024-05-09T20:27:52.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.556+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:52.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.549 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:52.551 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:52.552 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.560+0000 7f38e05ed980 20 searching for the correct realm 2024-05-09T20:27:52.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.564+0000 7f38de902640 0 -- 172.21.15.63:0/159393637 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f38d40da070 msgr2=0x7f38d40db400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:52.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.776+0000 7f38dd900640 0 -- 172.21.15.63:0/159393637 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f38d0056ea0 msgr2=0x7f38d0055b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:52.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.792+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:52.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:52.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 zone default found 2024-05-09T20:27:52.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 4 Realm: () 2024-05-09T20:27:52.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:52.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:52.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:52.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 zonegroup default 2024-05-09T20:27:52.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.796+0000 7f38e05ed980 20 started sync module instance, tier type = 2024-05-09T20:27:52.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.800+0000 7f38e05ed980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:52.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 20 add_watcher() i=0 2024-05-09T20:27:52.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 20 add_watcher() i=1 2024-05-09T20:27:52.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 20 add_watcher() i=2 2024-05-09T20:27:52.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 20 add_watcher() i=3 2024-05-09T20:27:52.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 20 add_watcher() i=4 2024-05-09T20:27:52.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 20 add_watcher() i=5 2024-05-09T20:27:52.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 20 add_watcher() i=6 2024-05-09T20:27:52.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 20 add_watcher() i=7 2024-05-09T20:27:52.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.824+0000 7f38e05ed980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:52.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.828+0000 7f3794fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:52.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:52.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:52.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 5 note: GC not initialized 2024-05-09T20:27:52.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f370f7fe640 20 reqs_thread_entry: start 2024-05-09T20:27:52.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:52.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 20 Filter name: none 2024-05-09T20:27:52.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f370d7fa640 20 reqs_thread_entry: start 2024-05-09T20:27:52.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 cache get: name=default.rgw.meta+root+abort-at-commit-target-layout : miss 2024-05-09T20:27:52.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:52.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 cache put: name=default.rgw.meta+root+abort-at-commit-target-layout info.flags=0x11 2024-05-09T20:27:52.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 adding default.rgw.meta+root+abort-at-commit-target-layout to cache LRU end 2024-05-09T20:27:52.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 : miss 2024-05-09T20:27:52.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:52.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:52.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:52.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:52.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:52.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.832+0000 7f38e05ed980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 2024-05-09T20:27:52.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.844+0000 7f38e05ed980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:52.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.844+0000 7f38e05ed980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:52.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.844+0000 7f38e05ed980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:52.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.844+0000 7f38e05ed980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26, ofs0, ns] 2024-05-09T20:27:52.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.844+0000 7f37c17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431600 cookie 94556116329696 notifier 6452 bl.length()=923 2024-05-09T20:27:52.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.844+0000 7f37c17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:52.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.844+0000 7f37c17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:52.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.844+0000 7f37c17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:52.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.848+0000 7f38e05ed980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_sources= new_sources= 2024-05-09T20:27:52.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.848+0000 7f38e05ed980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential sources added=[] removed=[] 2024-05-09T20:27:52.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.848+0000 7f38e05ed980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_dests= new_dests= 2024-05-09T20:27:52.838 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.848+0000 7f38e05ed980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential dests added=[] removed=[] 2024-05-09T20:27:52.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.0 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.0 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.1 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.1 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.2 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.2 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.3 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.3 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.4 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.4 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.5 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.5 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.6 2024-05-09T20:27:52.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.6 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.7 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.7 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.8 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.8 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.9 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.9 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.10 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.10 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.11 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1.11 2024-05-09T20:27:52.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.0 2024-05-09T20:27:52.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.1 2024-05-09T20:27:52.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.2 2024-05-09T20:27:52.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.852+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.3 2024-05-09T20:27:52.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.856+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.4 2024-05-09T20:27:52.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.856+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.5 2024-05-09T20:27:52.849 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.860+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.6 2024-05-09T20:27:52.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.860+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.7 2024-05-09T20:27:52.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.860+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.8 2024-05-09T20:27:52.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.860+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.9 2024-05-09T20:27:52.853 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.860+0000 7f38e05ed980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26.10 2024-05-09T20:27:52.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.868+0000 7f38e05ed980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 : hit (requested=0x17, cached=0x17) 2024-05-09T20:27:52.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.868+0000 7f38e05ed980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 2024-05-09T20:27:52.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.868+0000 7f38e05ed980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:52.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.868+0000 7f38e05ed980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:52.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.868+0000 7f38e05ed980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:52.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.868+0000 7f38e05ed980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26, ofs0, ns] 2024-05-09T20:27:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.872+0000 7f37c17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431601 cookie 94556116329696 notifier 6452 bl.length()=891 2024-05-09T20:27:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.872+0000 7f37c17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 info.flags=0x17 2024-05-09T20:27:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.872+0000 7f37c17fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-commit-target-layout:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 to cache LRU end 2024-05-09T20:27:52.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.872+0000 7f37c17fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:52.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.872+0000 7f38e05ed980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_sources= new_sources= 2024-05-09T20:27:52.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.872+0000 7f38e05ed980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential sources added=[] removed=[] 2024-05-09T20:27:52.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.872+0000 7f38e05ed980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): orig_dests= new_dests= 2024-05-09T20:27:52.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.872+0000 7f38e05ed980 20 handle_bi_update(): bucket=:abort-at-commit-target-layout[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26]): potential dests added=[] removed=[] 2024-05-09T20:27:52.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.880+0000 7f38e05ed980 1 execute INFO: reshard of bucket "abort-at-commit-target-layout" completed successfully 2024-05-09T20:27:52.873 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.884+0000 7f3794fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:52.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.888+0000 7f38e05ed980 20 remove_watcher() i=0 2024-05-09T20:27:52.878 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.888+0000 7f38e05ed980 2 removed watcher, disabling cache 2024-05-09T20:27:52.880 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.888+0000 7f38e05ed980 20 remove_watcher() i=1 2024-05-09T20:27:52.882 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.892+0000 7f38e05ed980 20 remove_watcher() i=2 2024-05-09T20:27:52.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.892+0000 7f38e05ed980 20 remove_watcher() i=3 2024-05-09T20:27:52.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.896+0000 7f38e05ed980 20 remove_watcher() i=4 2024-05-09T20:27:52.889 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.896+0000 7f38e05ed980 20 remove_watcher() i=5 2024-05-09T20:27:52.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.900+0000 7f38e05ed980 20 remove_watcher() i=6 2024-05-09T20:27:52.892 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.900+0000 7f38e05ed980 20 remove_watcher() i=7 2024-05-09T20:27:52.910 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-commit-target-layout 2024-05-09T20:27:52.942 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:52.943 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:52.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:52.996+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.989 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 realm 2024-05-09T20:27:52.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:52.990 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:52.991 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:52.992 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.000+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:52.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.004+0000 7f6d3149d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:52.993 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.004+0000 7f6d3149d980 20 searching for the correct realm 2024-05-09T20:27:52.996 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.004+0000 7f6d2f7b2640 0 -- 172.21.15.63:0/1911961809 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f6d24057420 msgr2=0x7f6d24057840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:53.209 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.216+0000 7f6d2e7b0640 0 -- 172.21.15.63:0/1911961809 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f6d20065880 msgr2=0x7f6d20075120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:53.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.232+0000 7f6d3149d980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.232+0000 7f6d3149d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.232+0000 7f6d3149d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.232+0000 7f6d3149d980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.232+0000 7f6d3149d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.232+0000 7f6d3149d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:53.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:53.227 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:53.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 zone default found 2024-05-09T20:27:53.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 4 Realm: () 2024-05-09T20:27:53.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:53.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:53.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:53.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 zonegroup default 2024-05-09T20:27:53.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:53.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.236+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.240+0000 7f6d3149d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:53.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.240+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.240+0000 7f6d3149d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:53.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.240+0000 7f6d3149d980 20 started sync module instance, tier type = 2024-05-09T20:27:53.230 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.240+0000 7f6d3149d980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.264+0000 7f6d3149d980 20 add_watcher() i=0 2024-05-09T20:27:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.264+0000 7f6d3149d980 20 add_watcher() i=1 2024-05-09T20:27:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.264+0000 7f6d3149d980 20 add_watcher() i=2 2024-05-09T20:27:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.264+0000 7f6d3149d980 20 add_watcher() i=3 2024-05-09T20:27:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.264+0000 7f6d3149d980 20 add_watcher() i=4 2024-05-09T20:27:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.264+0000 7f6d3149d980 20 add_watcher() i=5 2024-05-09T20:27:53.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.264+0000 7f6d3149d980 20 add_watcher() i=6 2024-05-09T20:27:53.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.268+0000 7f6d3149d980 20 add_watcher() i=7 2024-05-09T20:27:53.257 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.268+0000 7f6d3149d980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:53.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6be57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:53.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6d3149d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:53.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6d3149d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:53.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6d3149d980 5 note: GC not initialized 2024-05-09T20:27:53.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6b5ffff640 20 reqs_thread_entry: start 2024-05-09T20:27:53.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6d3149d980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:53.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6d3149d980 20 Filter name: none 2024-05-09T20:27:53.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6b5dffb640 20 reqs_thread_entry: start 2024-05-09T20:27:53.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6d3149d980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:53.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.272+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.276+0000 7f6d3149d980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:53.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.276+0000 7f6d3149d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:53.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.276+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.276+0000 7f6d3149d980 20 rados_obj.operate() r=0 bl.length=197 2024-05-09T20:27:53.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.276+0000 7f6d3149d980 20 rados->read ofs=0 len=0 2024-05-09T20:27:53.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.276+0000 7f6d3149d980 20 rados_obj.operate() r=0 bl.length=411 2024-05-09T20:27:53.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.276+0000 7f6d3149d980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:53.270 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.280+0000 7f6be57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:53.275 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.284+0000 7f6d3149d980 20 remove_watcher() i=0 2024-05-09T20:27:53.275 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.284+0000 7f6d3149d980 2 removed watcher, disabling cache 2024-05-09T20:27:53.277 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.288+0000 7f6d3149d980 20 remove_watcher() i=1 2024-05-09T20:27:53.279 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.288+0000 7f6d3149d980 20 remove_watcher() i=2 2024-05-09T20:27:53.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.292+0000 7f6d3149d980 20 remove_watcher() i=3 2024-05-09T20:27:53.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.292+0000 7f6d3149d980 20 remove_watcher() i=4 2024-05-09T20:27:53.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.292+0000 7f6d3149d980 20 remove_watcher() i=5 2024-05-09T20:27:53.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.296+0000 7f6d3149d980 20 remove_watcher() i=6 2024-05-09T20:27:53.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:53.296+0000 7f6d3149d980 20 remove_watcher() i=7 2024-05-09T20:27:53.304 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:53.304 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:53.304 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-commit-target-layout id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.26 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-09T20:27:53.305 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:54.212 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with EIO injected at do_reshard 2024-05-09T20:27:54.212 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:56.359 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-05-09T20:27:56.393 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:56.393 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:56.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 20 realm 2024-05-09T20:27:56.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:56.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.442 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.448+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.443 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.452+0000 7f9007160980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:56.443 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.452+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.452+0000 7f9007160980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:56.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.452+0000 7f9007160980 20 searching for the correct realm 2024-05-09T20:27:56.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.456+0000 7f9004473640 0 -- 172.21.15.63:0/3824412315 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f8ff405f360 msgr2=0x7f8ff4055ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:56.452 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.460+0000 7f9004c74640 0 -- 172.21.15.63:0/3824412315 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f8ffc027070 msgr2=0x7f8ffc0618d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:56.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:56.462 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:56.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:56.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:56.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:56.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:56.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.463 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:56.464 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.466 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 zone default found 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 4 Realm: () 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 zonegroup default 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 started sync module instance, tier type = 2024-05-09T20:27:56.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.472+0000 7f9007160980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:56.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.488+0000 7f9007160980 20 add_watcher() i=0 2024-05-09T20:27:56.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.488+0000 7f9007160980 20 add_watcher() i=1 2024-05-09T20:27:56.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.488+0000 7f9007160980 20 add_watcher() i=2 2024-05-09T20:27:56.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.488+0000 7f9007160980 20 add_watcher() i=3 2024-05-09T20:27:56.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.488+0000 7f9007160980 20 add_watcher() i=4 2024-05-09T20:27:56.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.488+0000 7f9007160980 20 add_watcher() i=5 2024-05-09T20:27:56.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.488+0000 7f9007160980 20 add_watcher() i=6 2024-05-09T20:27:56.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.492+0000 7f9007160980 20 add_watcher() i=7 2024-05-09T20:27:56.482 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.492+0000 7f9007160980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:56.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.492+0000 7f8ebafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:56.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.492+0000 7f9007160980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:56.485 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.492+0000 7f9007160980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:56.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.492+0000 7f9007160980 5 note: GC not initialized 2024-05-09T20:27:56.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f8eb9fe3640 20 reqs_thread_entry: start 2024-05-09T20:27:56.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:56.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 Filter name: none 2024-05-09T20:27:56.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f8e2b7fe640 20 reqs_thread_entry: start 2024-05-09T20:27:56.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.486 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:27:56.487 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:27:56.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:56.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:27:56.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.488 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:27:56.489 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.496+0000 7f9007160980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:56.491 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.500+0000 7f8ebafe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:56.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.504+0000 7f9007160980 20 remove_watcher() i=0 2024-05-09T20:27:56.496 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.504+0000 7f9007160980 2 removed watcher, disabling cache 2024-05-09T20:27:56.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.508+0000 7f9007160980 20 remove_watcher() i=1 2024-05-09T20:27:56.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.508+0000 7f9007160980 20 remove_watcher() i=2 2024-05-09T20:27:56.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.512+0000 7f9007160980 20 remove_watcher() i=3 2024-05-09T20:27:56.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.512+0000 7f9007160980 20 remove_watcher() i=4 2024-05-09T20:27:56.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.516+0000 7f9007160980 20 remove_watcher() i=5 2024-05-09T20:27:56.506 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.516+0000 7f9007160980 20 remove_watcher() i=6 2024-05-09T20:27:56.508 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.516+0000 7f9007160980 20 remove_watcher() i=7 2024-05-09T20:27:56.521 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:56.521 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:56.522 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-do-reshard id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:27:56.522 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:56.522 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-error-at do_reshard 2024-05-09T20:27:56.555 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:56.555 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:56.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 20 realm 2024-05-09T20:27:56.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:56.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:56.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.608+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.612+0000 7f86cd8f0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:56.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.612+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:56.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.612+0000 7f86cd8f0980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:56.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.612+0000 7f86cd8f0980 20 searching for the correct realm 2024-05-09T20:27:56.606 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:56.616+0000 7f86cb404640 0 -- 172.21.15.63:0/3815384160 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f86c400e070 msgr2=0x7f86c4090a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:57.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.036+0000 7f86cbc05640 0 -- 172.21.15.63:0/3815384160 >> [v2:172.21.15.63:6800/3125497649,v1:172.21.15.63:6801/3125497649] conn(0x55cec78b4a40 msgr2=0x55cec78b6e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:57.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:57.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 zone default found 2024-05-09T20:27:57.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 4 Realm: () 2024-05-09T20:27:57.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:57.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:57.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:57.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 zonegroup default 2024-05-09T20:27:57.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.048+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.052+0000 7f86cd8f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.052+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.052+0000 7f86cd8f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.042 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.052+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.052+0000 7f86cd8f0980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.052+0000 7f86cd8f0980 20 started sync module instance, tier type = 2024-05-09T20:27:57.045 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.052+0000 7f86cd8f0980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.080+0000 7f86cd8f0980 20 add_watcher() i=0 2024-05-09T20:27:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.080+0000 7f86cd8f0980 20 add_watcher() i=1 2024-05-09T20:27:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.080+0000 7f86cd8f0980 20 add_watcher() i=2 2024-05-09T20:27:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.080+0000 7f86cd8f0980 20 add_watcher() i=3 2024-05-09T20:27:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.080+0000 7f86cd8f0980 20 add_watcher() i=4 2024-05-09T20:27:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.080+0000 7f86cd8f0980 20 add_watcher() i=5 2024-05-09T20:27:57.072 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.080+0000 7f86cd8f0980 20 add_watcher() i=6 2024-05-09T20:27:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.084+0000 7f86cd8f0980 20 add_watcher() i=7 2024-05-09T20:27:57.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.084+0000 7f86cd8f0980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:57.078 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f8581fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:57.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f86cd8f0980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:57.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f86cd8f0980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:57.079 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f86cd8f0980 5 note: GC not initialized 2024-05-09T20:27:57.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f8580fe1640 20 reqs_thread_entry: start 2024-05-09T20:27:57.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f86cd8f0980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:57.080 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f86cd8f0980 20 Filter name: none 2024-05-09T20:27:57.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f84fa7fc640 20 reqs_thread_entry: start 2024-05-09T20:27:57.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f86cd8f0980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-05-09T20:27:57.081 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.088+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:27:57.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-05-09T20:27:57.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-05-09T20:27:57.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 : miss 2024-05-09T20:27:57.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:27:57.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.083 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:57.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.084 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.092+0000 7f86cd8f0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 2024-05-09T20:27:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.104+0000 7f86cd8f0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.104+0000 7f86cd8f0980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.104+0000 7f86cd8f0980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.104+0000 7f86cd8f0980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27, ofs0, ns] 2024-05-09T20:27:57.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.108+0000 7f85aeffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431636 cookie 94346599454880 notifier 6500 bl.length()=903 2024-05-09T20:27:57.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.108+0000 7f85aeffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.108+0000 7f85aeffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.108+0000 7f85aeffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.108+0000 7f86cd8f0980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): orig_sources= new_sources= 2024-05-09T20:27:57.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.108+0000 7f86cd8f0980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): potential sources added=[] removed=[] 2024-05-09T20:27:57.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.108+0000 7f86cd8f0980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): orig_dests= new_dests= 2024-05-09T20:27:57.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.108+0000 7f86cd8f0980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): potential dests added=[] removed=[] 2024-05-09T20:27:57.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.116+0000 7f86cd8f0980 -1 Injecting error=-5 at location=do_reshard 2024-05-09T20:27:57.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.128+0000 7f86cd8f0980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 : hit (requested=0x17, cached=0x17) 2024-05-09T20:27:57.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.128+0000 7f86cd8f0980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 2024-05-09T20:27:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.128+0000 7f86cd8f0980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.128+0000 7f86cd8f0980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.128+0000 7f86cd8f0980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.128+0000 7f86cd8f0980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27, ofs0, ns] 2024-05-09T20:27:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.132+0000 7f85ae7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431637 cookie 94346599454880 notifier 6500 bl.length()=871 2024-05-09T20:27:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.132+0000 7f85ae7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.132+0000 7f85ae7fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.132+0000 7f85ae7fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.136+0000 7f86cd8f0980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): orig_sources= new_sources= 2024-05-09T20:27:57.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.136+0000 7f86cd8f0980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): potential sources added=[] removed=[] 2024-05-09T20:27:57.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.136+0000 7f86cd8f0980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): orig_dests= new_dests= 2024-05-09T20:27:57.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.136+0000 7f86cd8f0980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): potential dests added=[] removed=[] 2024-05-09T20:27:57.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.136+0000 7f86cd8f0980 1 execute INFO: reshard of bucket "error-at-do-reshard" canceled due to errors 2024-05-09T20:27:57.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.140+0000 7f8581fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:57.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.144+0000 7f86cd8f0980 20 remove_watcher() i=0 2024-05-09T20:27:57.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.144+0000 7f86cd8f0980 2 removed watcher, disabling cache 2024-05-09T20:27:57.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.148+0000 7f86cd8f0980 20 remove_watcher() i=1 2024-05-09T20:27:57.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.148+0000 7f86cd8f0980 20 remove_watcher() i=2 2024-05-09T20:27:57.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.152+0000 7f86cd8f0980 20 remove_watcher() i=3 2024-05-09T20:27:57.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.152+0000 7f86cd8f0980 20 remove_watcher() i=4 2024-05-09T20:27:57.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.156+0000 7f86cd8f0980 20 remove_watcher() i=5 2024-05-09T20:27:57.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.156+0000 7f86cd8f0980 20 remove_watcher() i=6 2024-05-09T20:27:57.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.156+0000 7f86cd8f0980 20 remove_watcher() i=7 2024-05-09T20:27:57.164 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-05-09T20:27:57.193 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:57.193 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:57.235 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.244+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.237 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 realm 2024-05-09T20:27:57.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:57.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:57.239 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:57.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.248+0000 7f317196a980 20 searching for the correct realm 2024-05-09T20:27:57.243 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.252+0000 7f316fc7f640 0 -- 172.21.15.63:0/3535091064 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f316411f600 msgr2=0x7f31641280c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:57.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.260+0000 7f316ec7d640 0 -- 172.21.15.63:0/3535091064 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f3160008070 msgr2=0x7f31600567f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:57.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:57.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:57.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:57.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:57.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:57.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:57.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.262 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:57.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 zone default found 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 4 Realm: () 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 zonegroup default 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 started sync module instance, tier type = 2024-05-09T20:27:57.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.272+0000 7f317196a980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:57.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 20 add_watcher() i=0 2024-05-09T20:27:57.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 20 add_watcher() i=1 2024-05-09T20:27:57.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 20 add_watcher() i=2 2024-05-09T20:27:57.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 20 add_watcher() i=3 2024-05-09T20:27:57.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 20 add_watcher() i=4 2024-05-09T20:27:57.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 20 add_watcher() i=5 2024-05-09T20:27:57.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 20 add_watcher() i=6 2024-05-09T20:27:57.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 20 add_watcher() i=7 2024-05-09T20:27:57.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.296+0000 7f317196a980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:57.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f3025fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:57.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f317196a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:57.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f317196a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:57.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f317196a980 5 note: GC not initialized 2024-05-09T20:27:57.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f3024fe1640 20 reqs_thread_entry: start 2024-05-09T20:27:57.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f317196a980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:57.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f317196a980 20 Filter name: none 2024-05-09T20:27:57.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f2f9e7fc640 20 reqs_thread_entry: start 2024-05-09T20:27:57.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f317196a980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:27:57.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.300+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.304+0000 7f317196a980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:27:57.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.304+0000 7f317196a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:57.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.304+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.304+0000 7f317196a980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:27:57.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.304+0000 7f317196a980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.304+0000 7f317196a980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:27:57.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.304+0000 7f317196a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:57.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.308+0000 7f3025fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:57.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.312+0000 7f317196a980 20 remove_watcher() i=0 2024-05-09T20:27:57.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.312+0000 7f317196a980 2 removed watcher, disabling cache 2024-05-09T20:27:57.304 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.312+0000 7f317196a980 20 remove_watcher() i=1 2024-05-09T20:27:57.306 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.316+0000 7f317196a980 20 remove_watcher() i=2 2024-05-09T20:27:57.309 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.316+0000 7f317196a980 20 remove_watcher() i=3 2024-05-09T20:27:57.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.320+0000 7f317196a980 20 remove_watcher() i=4 2024-05-09T20:27:57.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.320+0000 7f317196a980 20 remove_watcher() i=5 2024-05-09T20:27:57.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.324+0000 7f317196a980 20 remove_watcher() i=6 2024-05-09T20:27:57.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.324+0000 7f317196a980 20 remove_watcher() i=7 2024-05-09T20:27:57.329 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:57.329 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:57.329 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-do-reshard id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:27:57.329 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:57.435 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket error-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:27:57.470 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:57.471 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:57.514 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 20 realm 2024-05-09T20:27:57.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:57.516 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.517 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.524+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.528+0000 7f25a5d6e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:57.518 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.528+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.528+0000 7f25a5d6e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:57.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.528+0000 7f25a5d6e980 20 searching for the correct realm 2024-05-09T20:27:57.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.532+0000 7f25a3882640 0 -- 172.21.15.63:0/3751541514 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f259c034460 msgr2=0x7f259c0348e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:57.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.748+0000 7f25a3882640 0 -- 172.21.15.63:0/3751541514 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f259c034460 msgr2=0x7f259c0348e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:57.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:57.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:57.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:57.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:57.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:57.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:57.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.760+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:57.753 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:57.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 zone default found 2024-05-09T20:27:57.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 4 Realm: () 2024-05-09T20:27:57.754 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:57.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:57.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:57.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 zonegroup default 2024-05-09T20:27:57.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 started sync module instance, tier type = 2024-05-09T20:27:57.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.764+0000 7f25a5d6e980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:57.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.788+0000 7f25a5d6e980 20 add_watcher() i=0 2024-05-09T20:27:57.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.788+0000 7f25a5d6e980 20 add_watcher() i=1 2024-05-09T20:27:57.780 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.788+0000 7f25a5d6e980 20 add_watcher() i=2 2024-05-09T20:27:57.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.788+0000 7f25a5d6e980 20 add_watcher() i=3 2024-05-09T20:27:57.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.788+0000 7f25a5d6e980 20 add_watcher() i=4 2024-05-09T20:27:57.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.788+0000 7f25a5d6e980 20 add_watcher() i=5 2024-05-09T20:27:57.781 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.788+0000 7f25a5d6e980 20 add_watcher() i=6 2024-05-09T20:27:57.782 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.792+0000 7f25a5d6e980 20 add_watcher() i=7 2024-05-09T20:27:57.782 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.792+0000 7f25a5d6e980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:57.786 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f2459fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:57.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f25a5d6e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:57.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f25a5d6e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:57.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f25a5d6e980 5 note: GC not initialized 2024-05-09T20:27:57.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f2458fe1640 20 reqs_thread_entry: start 2024-05-09T20:27:57.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f25a5d6e980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:57.788 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f25a5d6e980 20 Filter name: none 2024-05-09T20:27:57.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f23d27fc640 20 reqs_thread_entry: start 2024-05-09T20:27:57.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f25a5d6e980 10 cache get: name=default.rgw.meta+root+error-at-do-reshard : miss 2024-05-09T20:27:57.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.796+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:27:57.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 10 cache put: name=default.rgw.meta+root+error-at-do-reshard info.flags=0x11 2024-05-09T20:27:57.789 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 10 adding default.rgw.meta+root+error-at-do-reshard to cache LRU end 2024-05-09T20:27:57.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 : miss 2024-05-09T20:27:57.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.790 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:27:57.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 10 adding default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:27:57.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.800+0000 7f25a5d6e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 2024-05-09T20:27:57.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.812+0000 7f25a5d6e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.803 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.812+0000 7f25a5d6e980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.812+0000 7f25a5d6e980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.812+0000 7f25a5d6e980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27, ofs0, ns] 2024-05-09T20:27:57.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.812+0000 7f2482ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431638 cookie 93996183259664 notifier 6927 bl.length()=903 2024-05-09T20:27:57.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.812+0000 7f2482ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.812+0000 7f2482ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.812+0000 7f2482ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.816+0000 7f25a5d6e980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): orig_sources= new_sources= 2024-05-09T20:27:57.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.816+0000 7f25a5d6e980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): potential sources added=[] removed=[] 2024-05-09T20:27:57.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.816+0000 7f25a5d6e980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): orig_dests= new_dests= 2024-05-09T20:27:57.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.816+0000 7f25a5d6e980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): potential dests added=[] removed=[] 2024-05-09T20:27:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.0 2024-05-09T20:27:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.0 2024-05-09T20:27:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.1 2024-05-09T20:27:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.1 2024-05-09T20:27:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.2 2024-05-09T20:27:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.2 2024-05-09T20:27:57.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.3 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.3 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.4 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.4 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.5 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.5 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.6 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.6 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.7 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.7 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.8 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.8 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.9 2024-05-09T20:27:57.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.9 2024-05-09T20:27:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.10 2024-05-09T20:27:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.10 2024-05-09T20:27:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.11 2024-05-09T20:27:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1.11 2024-05-09T20:27:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.0 2024-05-09T20:27:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.1 2024-05-09T20:27:57.816 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.2 2024-05-09T20:27:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.3 2024-05-09T20:27:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.824+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.4 2024-05-09T20:27:57.817 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.828+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.5 2024-05-09T20:27:57.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.828+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.6 2024-05-09T20:27:57.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.828+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.7 2024-05-09T20:27:57.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.828+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.8 2024-05-09T20:27:57.820 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.828+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.9 2024-05-09T20:27:57.822 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.832+0000 7f25a5d6e980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27.10 2024-05-09T20:27:57.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.836+0000 7f25a5d6e980 10 cache get: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 : hit (requested=0x17, cached=0x17) 2024-05-09T20:27:57.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.836+0000 7f25a5d6e980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 2024-05-09T20:27:57.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.840+0000 7f25a5d6e980 10 cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.840+0000 7f25a5d6e980 10 moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.840+0000 7f25a5d6e980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.840+0000 7f25a5d6e980 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27, ofs0, ns] 2024-05-09T20:27:57.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.844+0000 7f24827fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431639 cookie 93996183259664 notifier 6927 bl.length()=871 2024-05-09T20:27:57.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.844+0000 7f24827fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 info.flags=0x17 2024-05-09T20:27:57.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.844+0000 7f24827fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.error-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 to cache LRU end 2024-05-09T20:27:57.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.844+0000 7f24827fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:27:57.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.844+0000 7f25a5d6e980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): orig_sources= new_sources= 2024-05-09T20:27:57.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.844+0000 7f25a5d6e980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): potential sources added=[] removed=[] 2024-05-09T20:27:57.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.844+0000 7f25a5d6e980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): orig_dests= new_dests= 2024-05-09T20:27:57.836 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.844+0000 7f25a5d6e980 20 handle_bi_update(): bucket=:error-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27]): potential dests added=[] removed=[] 2024-05-09T20:27:57.840 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.848+0000 7f25a5d6e980 1 execute INFO: reshard of bucket "error-at-do-reshard" completed successfully 2024-05-09T20:27:57.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.852+0000 7f2459fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:57.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.856+0000 7f25a5d6e980 20 remove_watcher() i=0 2024-05-09T20:27:57.848 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.856+0000 7f25a5d6e980 2 removed watcher, disabling cache 2024-05-09T20:27:57.850 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.860+0000 7f25a5d6e980 20 remove_watcher() i=1 2024-05-09T20:27:57.852 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.860+0000 7f25a5d6e980 20 remove_watcher() i=2 2024-05-09T20:27:57.854 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.864+0000 7f25a5d6e980 20 remove_watcher() i=3 2024-05-09T20:27:57.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.864+0000 7f25a5d6e980 20 remove_watcher() i=4 2024-05-09T20:27:57.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.868+0000 7f25a5d6e980 20 remove_watcher() i=5 2024-05-09T20:27:57.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.868+0000 7f25a5d6e980 20 remove_watcher() i=6 2024-05-09T20:27:57.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.872+0000 7f25a5d6e980 20 remove_watcher() i=7 2024-05-09T20:27:57.877 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket error-at-do-reshard 2024-05-09T20:27:57.903 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:27:57.903 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:27:57.947 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.956+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.956+0000 7f91a9d96980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.956+0000 7f91a9d96980 20 realm 2024-05-09T20:27:57.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.956+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.956+0000 7f91a9d96980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.956+0000 7f91a9d96980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:27:57.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.956+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.960+0000 7f91a9d96980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:57.949 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.960+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.960+0000 7f91a9d96980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:57.951 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.960+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:57.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.960+0000 7f91a9d96980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:27:57.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.960+0000 7f91a9d96980 20 searching for the correct realm 2024-05-09T20:27:57.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:57.964+0000 7f91a78aa640 0 -- 172.21.15.63:0/1170910164 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f91a00d7060 msgr2=0x7f91a00fb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:27:58.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.180+0000 7f91a78aa640 0 -- 172.21.15.63:0/1170910164 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f91a00d7060 msgr2=0x7f91a00fb6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:58.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:27:58.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.192+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:27:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 zone default found 2024-05-09T20:27:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 4 Realm: () 2024-05-09T20:27:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:27:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:27:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:27:58.186 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 zonegroup default 2024-05-09T20:27:58.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:58.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:58.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:27:58.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 started sync module instance, tier type = 2024-05-09T20:27:58.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.196+0000 7f91a9d96980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:27:58.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.220+0000 7f91a9d96980 20 add_watcher() i=0 2024-05-09T20:27:58.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.220+0000 7f91a9d96980 20 add_watcher() i=1 2024-05-09T20:27:58.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.220+0000 7f91a9d96980 20 add_watcher() i=2 2024-05-09T20:27:58.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.220+0000 7f91a9d96980 20 add_watcher() i=3 2024-05-09T20:27:58.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.220+0000 7f91a9d96980 20 add_watcher() i=4 2024-05-09T20:27:58.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.220+0000 7f91a9d96980 20 add_watcher() i=5 2024-05-09T20:27:58.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.220+0000 7f91a9d96980 20 add_watcher() i=6 2024-05-09T20:27:58.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.224+0000 7f91a9d96980 20 add_watcher() i=7 2024-05-09T20:27:58.215 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.224+0000 7f91a9d96980 2 all 8 watchers are set, enabling cache 2024-05-09T20:27:58.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.228+0000 7f905dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:58.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.228+0000 7f91a9d96980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:27:58.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.228+0000 7f91a9d96980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:27:58.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.228+0000 7f91a9d96980 5 note: GC not initialized 2024-05-09T20:27:58.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.228+0000 7f905cfe1640 20 reqs_thread_entry: start 2024-05-09T20:27:58.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.228+0000 7f91a9d96980 20 init_complete bucket index max shards: 11 2024-05-09T20:27:58.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.228+0000 7f91a9d96980 20 Filter name: none 2024-05-09T20:27:58.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.228+0000 7f8fd67fc640 20 reqs_thread_entry: start 2024-05-09T20:27:58.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:27:58.222 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:27:58.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:58.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:27:58.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 rados->read ofs=0 len=0 2024-05-09T20:27:58.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:27:58.225 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.232+0000 7f91a9d96980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:27:58.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.236+0000 7f905dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:27:58.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.244+0000 7f91a9d96980 20 remove_watcher() i=0 2024-05-09T20:27:58.234 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.244+0000 7f91a9d96980 2 removed watcher, disabling cache 2024-05-09T20:27:58.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.244+0000 7f91a9d96980 20 remove_watcher() i=1 2024-05-09T20:27:58.238 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.248+0000 7f91a9d96980 20 remove_watcher() i=2 2024-05-09T20:27:58.240 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.248+0000 7f91a9d96980 20 remove_watcher() i=3 2024-05-09T20:27:58.242 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.252+0000 7f91a9d96980 20 remove_watcher() i=4 2024-05-09T20:27:58.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.252+0000 7f91a9d96980 20 remove_watcher() i=5 2024-05-09T20:27:58.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.256+0000 7f91a9d96980 20 remove_watcher() i=6 2024-05-09T20:27:58.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:27:58.256+0000 7f91a9d96980 20 remove_watcher() i=7 2024-05-09T20:27:58.262 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:58.262 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:27:58.263 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: error-at-do-reshard id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.27 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-09T20:27:58.263 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:27:59.149 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: reshard bucket with abort at do_reshard 2024-05-09T20:27:59.149 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:01.248 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-05-09T20:28:01.280 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:01.281 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:01.326 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 20 realm 2024-05-09T20:28:01.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:01.329 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.336+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.340+0000 7fc1f6c3b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:01.330 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.340+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.340+0000 7fc1f6c3b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:01.331 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.340+0000 7fc1f6c3b980 20 searching for the correct realm 2024-05-09T20:28:01.333 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.340+0000 7fc1f3f4e640 0 -- 172.21.15.63:0/2339332346 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fc1dc0022b0 msgr2=0x7fc1e4034420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:01.543 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.552+0000 7fc1f3f4e640 0 -- 172.21.15.63:0/2339332346 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7fc1e4045930 msgr2=0x7fc1e4043400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:01.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:01.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.776+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:01.769 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 zone default found 2024-05-09T20:28:01.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 4 Realm: () 2024-05-09T20:28:01.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:01.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:01.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:01.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 zonegroup default 2024-05-09T20:28:01.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.770 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.771 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 started sync module instance, tier type = 2024-05-09T20:28:01.772 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.780+0000 7fc1f6c3b980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:01.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.804+0000 7fc1f6c3b980 20 add_watcher() i=0 2024-05-09T20:28:01.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.804+0000 7fc1f6c3b980 20 add_watcher() i=1 2024-05-09T20:28:01.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.804+0000 7fc1f6c3b980 20 add_watcher() i=2 2024-05-09T20:28:01.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.804+0000 7fc1f6c3b980 20 add_watcher() i=3 2024-05-09T20:28:01.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.804+0000 7fc1f6c3b980 20 add_watcher() i=4 2024-05-09T20:28:01.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.804+0000 7fc1f6c3b980 20 add_watcher() i=5 2024-05-09T20:28:01.797 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.804+0000 7fc1f6c3b980 20 add_watcher() i=6 2024-05-09T20:28:01.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.808+0000 7fc1f6c3b980 20 add_watcher() i=7 2024-05-09T20:28:01.800 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.808+0000 7fc1f6c3b980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:01.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.812+0000 7fc0aafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:01.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:01.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:01.805 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 5 note: GC not initialized 2024-05-09T20:28:01.806 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc0a9fdb640 20 reqs_thread_entry: start 2024-05-09T20:28:01.806 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:01.806 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 Filter name: none 2024-05-09T20:28:01.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc01b7fe640 20 reqs_thread_entry: start 2024-05-09T20:28:01.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:01.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:28:01.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.816+0000 7fc1f6c3b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:01.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.820+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.820+0000 7fc1f6c3b980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:01.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.820+0000 7fc1f6c3b980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.820+0000 7fc1f6c3b980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:28:01.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.820+0000 7fc1f6c3b980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:01.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.824+0000 7fc0aafdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:01.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.828+0000 7fc1f6c3b980 20 remove_watcher() i=0 2024-05-09T20:28:01.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.828+0000 7fc1f6c3b980 2 removed watcher, disabling cache 2024-05-09T20:28:01.821 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.832+0000 7fc1f6c3b980 20 remove_watcher() i=1 2024-05-09T20:28:01.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.832+0000 7fc1f6c3b980 20 remove_watcher() i=2 2024-05-09T20:28:01.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.836+0000 7fc1f6c3b980 20 remove_watcher() i=3 2024-05-09T20:28:01.828 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.836+0000 7fc1f6c3b980 20 remove_watcher() i=4 2024-05-09T20:28:01.829 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.840+0000 7fc1f6c3b980 20 remove_watcher() i=5 2024-05-09T20:28:01.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.840+0000 7fc1f6c3b980 20 remove_watcher() i=6 2024-05-09T20:28:01.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.840+0000 7fc1f6c3b980 20 remove_watcher() i=7 2024-05-09T20:28:01.850 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:01.850 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:28:01.850 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-do-reshard id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:28:01.850 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:01.851 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 --inject-abort-at do_reshard 2024-05-09T20:28:01.884 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:01.884 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:01.927 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.936+0000 7f67f62cd980 20 realm 2024-05-09T20:28:01.929 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:01.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.930 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:01.931 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:01.932 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.940+0000 7f67f62cd980 20 searching for the correct realm 2024-05-09T20:28:01.935 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.944+0000 7f67f45e2640 0 -- 172.21.15.63:0/2688707805 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f67e8108040 msgr2=0x7f67e80dc360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:01.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.952+0000 7f67f35e0640 0 -- 172.21.15.63:0/2688707805 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f67e4039940 msgr2=0x7f67e400f3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.952 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:01.954 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:01.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 zone default found 2024-05-09T20:28:01.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 4 Realm: () 2024-05-09T20:28:01.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:01.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:01.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:01.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 zonegroup default 2024-05-09T20:28:01.955 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:01.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.964+0000 7f67f62cd980 20 started sync module instance, tier type = 2024-05-09T20:28:01.956 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.964+0000 7f67f62cd980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:01.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=0 2024-05-09T20:28:01.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=1 2024-05-09T20:28:01.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=2 2024-05-09T20:28:01.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=3 2024-05-09T20:28:01.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=4 2024-05-09T20:28:01.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=5 2024-05-09T20:28:01.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=6 2024-05-09T20:28:01.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=7 2024-05-09T20:28:01.971 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.980+0000 7f67f62cd980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:01.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f66aa7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:01.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:01.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:01.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 5 note: GC not initialized 2024-05-09T20:28:01.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f66a97e2640 20 reqs_thread_entry: start 2024-05-09T20:28:01.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:01.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 20 Filter name: none 2024-05-09T20:28:01.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f66a27fc640 20 reqs_thread_entry: start 2024-05-09T20:28:01.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-05-09T20:28:01.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:01.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-05-09T20:28:01.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-05-09T20:28:01.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 : miss 2024-05-09T20:28:01.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:01.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:28:01.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:01.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:01.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:01.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:01.978 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.984+0000 7f67f62cd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 2024-05-09T20:28:01.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.992+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:01.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.992+0000 7f67f62cd980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:01.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.992+0000 7f67f62cd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:01.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.992+0000 7f67f62cd980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28, ofs0, ns] 2024-05-09T20:28:01.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431610 cookie 94421344665952 notifier 6978 bl.length()=903 2024-05-09T20:28:01.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:01.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:01.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:01.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_sources= new_sources= 2024-05-09T20:28:01.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential sources added=[] removed=[] 2024-05-09T20:28:01.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_dests= new_dests= 2024-05-09T20:28:01.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential dests added=[] removed=[] 2024-05-09T20:28:01.993 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f67f62cd980 time 2024-05-09T20:28:02.005601+0000 2024-05-09T20:28:01.993 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:28:01.993 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f67f9ba3e25] 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f67f9ba3fd9] 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e02c5b3ca1] 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31d) [0x55e02c5c441d] 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f67f8a0dd90] 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.004+0000 7f67f62cd980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f67f62cd980 time 2024-05-09T20:28:02.005601+0000 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:28:01.994 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f67f9ba3e25] 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f67f9ba3fd9] 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e02c5b3ca1] 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31d) [0x55e02c5c441d] 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f67f8a0dd90] 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr:*** Caught signal (Aborted) ** 2024-05-09T20:28:01.995 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f67f62cd980 thread_name:radosgw-admin 2024-05-09T20:28:02.020 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:28:02.020 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f67f8a26520] 2024-05-09T20:28:02.020 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:28:02.020 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:28:02.020 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:28:02.020 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f67f9ba3e7f] 2024-05-09T20:28:02.020 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f67f9ba3fd9] 2024-05-09T20:28:02.020 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e02c5b3ca1] 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31d) [0x55e02c5c441d] 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f67f8a0dd90] 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.004+0000 7f67f62cd980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f67f62cd980 thread_name:radosgw-admin 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.021 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:28:02.022 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f67f8a26520] 2024-05-09T20:28:02.022 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:28:02.022 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:28:02.022 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:28:02.022 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f67f9ba3e7f] 2024-05-09T20:28:02.022 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f67f9ba3fd9] 2024-05-09T20:28:02.022 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e02c5b3ca1] 2024-05-09T20:28:02.022 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31d) [0x55e02c5c441d] 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f67f8a0dd90] 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr:--- begin dump of recent events --- 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: -499> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: -498> 2024-05-09T20:28:01.916+0000 7f67f35e0640 1 monclient: found mon.noname-c 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: -497> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: -496> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.023 INFO:tasks.workunit.client.0.smithi063.stderr: -495> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr: -494> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: dump: 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.024 INFO:tasks.workunit.client.0.smithi063.stderr: -493> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr: -492> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr: -491> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr: -490> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr: -489> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr: -488> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: dump: 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.025 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.026 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.026 INFO:tasks.workunit.client.0.smithi063.stderr: -487> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6572 2024-05-09T20:28:02.026 INFO:tasks.workunit.client.0.smithi063.stderr: -486> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: get_monmap_and_config success 2024-05-09T20:28:02.026 INFO:tasks.workunit.client.0.smithi063.stderr: -485> 2024-05-09T20:28:01.916+0000 7f67f62cd980 4 set_mon_vals no callback set 2024-05-09T20:28:02.026 INFO:tasks.workunit.client.0.smithi063.stderr: -484> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: shutdown 2024-05-09T20:28:02.026 INFO:tasks.workunit.client.0.smithi063.stderr: -483> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 asok(0x55e02e84b430) unregister_commands rotate-key 2024-05-09T20:28:02.026 INFO:tasks.workunit.client.0.smithi063.stderr: -482> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: build_initial_monmap 2024-05-09T20:28:02.027 INFO:tasks.workunit.client.0.smithi063.stderr: -481> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 build_initial for_mkfs: 0 2024-05-09T20:28:02.027 INFO:tasks.workunit.client.0.smithi063.stderr: -480> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: monmap: 2024-05-09T20:28:02.027 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:28:02.027 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:28:02.027 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:28:02.027 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:28:02.027 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr: -479> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding auth protocol: cephx 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr: -478> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding auth protocol: cephx 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr: -477> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding auth protocol: cephx 2024-05-09T20:28:02.028 INFO:tasks.workunit.client.0.smithi063.stderr: -476> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.029 INFO:tasks.workunit.client.0.smithi063.stderr: -475> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.029 INFO:tasks.workunit.client.0.smithi063.stderr: -474> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.029 INFO:tasks.workunit.client.0.smithi063.stderr: -473> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.029 INFO:tasks.workunit.client.0.smithi063.stderr: -472> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.029 INFO:tasks.workunit.client.0.smithi063.stderr: -471> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.029 INFO:tasks.workunit.client.0.smithi063.stderr: -470> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.029 INFO:tasks.workunit.client.0.smithi063.stderr: -469> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.029 INFO:tasks.workunit.client.0.smithi063.stderr: -468> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -467> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -466> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -465> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -464> 2024-05-09T20:28:01.916+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -463> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 librados: starting msgr at 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -462> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 librados: starting objecter 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -461> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command objecter_requests hook 0x55e02e8194c0 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -460> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 librados: setting wanted keys 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -459> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 librados: calling monclient init 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -458> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: init 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -457> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding auth protocol: cephx 2024-05-09T20:28:02.030 INFO:tasks.workunit.client.0.smithi063.stderr: -456> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding auth protocol: cephx 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -455> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding auth protocol: cephx 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -454> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -453> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -452> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -451> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -450> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -449> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -448> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -447> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -446> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.031 INFO:tasks.workunit.client.0.smithi063.stderr: -445> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -444> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -443> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -442> 2024-05-09T20:28:01.916+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -441> 2024-05-09T20:28:01.916+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -440> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command rotate-key hook 0x55e02ea54c08 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -439> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: _reopen_session rank -1 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -438> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -437> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-b con 0x55e02e906360 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -436> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-a con 0x55e02e910d70 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -435> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-c con 0x55e02e99bb40 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -434> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.032 INFO:tasks.workunit.client.0.smithi063.stderr: -433> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -432> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -431> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): _renew_subs 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -430> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): authenticate will time out at 2089.059621s 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -429> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): get_auth_request con 0x55e02e910d70 auth_method 0 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -428> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -427> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -426> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -425> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): get_auth_request con 0x55e02e99bb40 auth_method 0 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -424> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -423> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -422> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -421> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.033 INFO:tasks.workunit.client.0.smithi063.stderr: -420> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -419> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -418> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -417> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -416> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -415> 2024-05-09T20:28:01.916+0000 7f67f3de1640 10 monclient(hunting): get_auth_request con 0x55e02e906360 auth_method 0 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -414> 2024-05-09T20:28:01.916+0000 7f67f3de1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -413> 2024-05-09T20:28:01.916+0000 7f67f3de1640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -412> 2024-05-09T20:28:01.916+0000 7f67f3de1640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -411> 2024-05-09T20:28:01.920+0000 7f67f45e2640 10 monclient(hunting): handle_auth_done global_id 6575 payload 824 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -410> 2024-05-09T20:28:01.920+0000 7f67f45e2640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.034 INFO:tasks.workunit.client.0.smithi063.stderr: -409> 2024-05-09T20:28:01.920+0000 7f67f45e2640 1 monclient: found mon.noname-c 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr: -408> 2024-05-09T20:28:01.920+0000 7f67f45e2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr: -407> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr: -406> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr: -405> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: dump: 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.035 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr: -404> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr: -403> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr: -402> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr: -401> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr: -400> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr: -399> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: dump: 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.036 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: -398> 2024-05-09T20:28:01.920+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6575 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: -397> 2024-05-09T20:28:01.920+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: -396> 2024-05-09T20:28:01.920+0000 7f66e1e0b640 4 set_mon_vals no callback set 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: -395> 2024-05-09T20:28:01.920+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: -394> 2024-05-09T20:28:01.920+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: -393> 2024-05-09T20:28:01.920+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: -392> 2024-05-09T20:28:01.920+0000 7f67f62cd980 1 librados: init done 2024-05-09T20:28:02.037 INFO:tasks.workunit.client.0.smithi063.stderr: -391> 2024-05-09T20:28:01.920+0000 7f66dbfff640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -390> 2024-05-09T20:28:01.920+0000 7f66dbfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -389> 2024-05-09T20:28:01.920+0000 7f66dbfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -388> 2024-05-09T20:28:01.920+0000 7f67f3de1640 10 monclient: get_auth_request con 0x7f66dc066d60 auth_method 0 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -387> 2024-05-09T20:28:01.920+0000 7f67f35e0640 10 monclient: get_auth_request con 0x55e02ea7a1f0 auth_method 0 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -386> 2024-05-09T20:28:01.924+0000 7f67f3de1640 10 monclient: get_auth_request con 0x55e02ea9fce0 auth_method 0 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -385> 2024-05-09T20:28:01.924+0000 7f67f35e0640 10 monclient: get_auth_request con 0x55e02eaa3ed0 auth_method 0 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -384> 2024-05-09T20:28:01.924+0000 7f67f45e2640 10 monclient: get_auth_request con 0x55e02eaa35c0 auth_method 0 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -383> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: build_initial_monmap 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -382> 2024-05-09T20:28:01.928+0000 7f67f62cd980 1 build_initial for_mkfs: 0 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr: -381> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: monmap: 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:28:02.038 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr: -380> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding auth protocol: cephx 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr: -379> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding auth protocol: cephx 2024-05-09T20:28:02.039 INFO:tasks.workunit.client.0.smithi063.stderr: -378> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding auth protocol: cephx 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -377> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -376> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -375> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -374> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -373> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -372> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -371> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -370> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -369> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -368> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -367> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.040 INFO:tasks.workunit.client.0.smithi063.stderr: -366> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -365> 2024-05-09T20:28:01.928+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -364> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command objecter_requests cmddesc objecter_requests hook 0x55e02eab1560 EEXIST 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -363> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: init 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -362> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding auth protocol: cephx 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -361> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding auth protocol: cephx 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -360> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding auth protocol: cephx 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -359> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -358> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -357> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -356> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -355> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.041 INFO:tasks.workunit.client.0.smithi063.stderr: -354> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -353> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -352> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -351> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -350> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -349> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -348> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -347> 2024-05-09T20:28:01.928+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -346> 2024-05-09T20:28:01.928+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -345> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command rotate-key cmddesc rotate-key hook 0x55e02eaab050 EEXIST 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -344> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _reopen_session rank -1 2024-05-09T20:28:02.042 INFO:tasks.workunit.client.0.smithi063.stderr: -343> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -342> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-c con 0x55e02ead61c0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -341> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-a con 0x55e02ead65a0 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -340> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-b con 0x55e02ead1760 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -339> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -338> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -337> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -336> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): _renew_subs 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -335> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): authenticate will time out at 2089.069228s 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -334> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): get_auth_request con 0x55e02ead61c0 auth_method 0 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -333> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -332> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -331> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -330> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): get_auth_request con 0x55e02ead65a0 auth_method 0 2024-05-09T20:28:02.043 INFO:tasks.workunit.client.0.smithi063.stderr: -329> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -328> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -327> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -326> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -325> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -324> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -323> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -322> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -321> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -320> 2024-05-09T20:28:01.928+0000 7f67f3de1640 10 monclient(hunting): get_auth_request con 0x55e02ead1760 auth_method 0 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -319> 2024-05-09T20:28:01.928+0000 7f67f3de1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -318> 2024-05-09T20:28:01.928+0000 7f67f3de1640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -317> 2024-05-09T20:28:01.928+0000 7f67f3de1640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.044 INFO:tasks.workunit.client.0.smithi063.stderr: -316> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): handle_auth_done global_id 6578 payload 824 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr: -315> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr: -314> 2024-05-09T20:28:01.928+0000 7f67f35e0640 1 monclient: found mon.noname-c 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr: -313> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr: -312> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr: -311> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr: -310> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: dump: 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.045 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr: -309> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr: -308> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr: -307> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr: -306> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr: -305> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr: -304> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: dump: 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.046 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr: -303> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6578 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr: -302> 2024-05-09T20:28:01.928+0000 7f67f4de3640 4 set_mon_vals no callback set 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr: -301> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr: -300> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.047 INFO:tasks.workunit.client.0.smithi063.stderr: -299> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -298> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -297> 2024-05-09T20:28:01.928+0000 7f66d97fa640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -296> 2024-05-09T20:28:01.928+0000 7f66d97fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -295> 2024-05-09T20:28:01.928+0000 7f66d97fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -294> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient: get_auth_request con 0x7f66cc066ca0 auth_method 0 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -293> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: get_monmap_and_config 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -292> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: build_initial_monmap 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -291> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 build_initial for_mkfs: 0 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr: -290> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: monmap: 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:28:02.048 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr: -289> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr: -288> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr: -287> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.049 INFO:tasks.workunit.client.0.smithi063.stderr: -286> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.050 INFO:tasks.workunit.client.0.smithi063.stderr: -285> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.050 INFO:tasks.workunit.client.0.smithi063.stderr: -284> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.050 INFO:tasks.workunit.client.0.smithi063.stderr: -283> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.050 INFO:tasks.workunit.client.0.smithi063.stderr: -282> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.050 INFO:tasks.workunit.client.0.smithi063.stderr: -281> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.050 INFO:tasks.workunit.client.0.smithi063.stderr: -280> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.050 INFO:tasks.workunit.client.0.smithi063.stderr: -279> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.050 INFO:tasks.workunit.client.0.smithi063.stderr: -278> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -277> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -276> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -275> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -274> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -273> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: init 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -272> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding auth protocol: cephx 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -271> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding auth protocol: cephx 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -270> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding auth protocol: cephx 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -269> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -268> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -267> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.051 INFO:tasks.workunit.client.0.smithi063.stderr: -266> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.052 INFO:tasks.workunit.client.0.smithi063.stderr: -265> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.052 INFO:tasks.workunit.client.0.smithi063.stderr: -264> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.052 INFO:tasks.workunit.client.0.smithi063.stderr: -263> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.052 INFO:tasks.workunit.client.0.smithi063.stderr: -262> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.052 INFO:tasks.workunit.client.0.smithi063.stderr: -261> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.052 INFO:tasks.workunit.client.0.smithi063.stderr: -260> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.052 INFO:tasks.workunit.client.0.smithi063.stderr: -259> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.052 INFO:tasks.workunit.client.0.smithi063.stderr: -258> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -257> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -256> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -255> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command rotate-key cmddesc rotate-key hook 0x7ffde2ebbb18 EEXIST 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -254> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: _reopen_session rank -1 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -253> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -252> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-a con 0x55e02eb28250 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -251> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-b con 0x55e02eb22860 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -250> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-c con 0x55e02eaf92d0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -249> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -248> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.053 INFO:tasks.workunit.client.0.smithi063.stderr: -247> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.054 INFO:tasks.workunit.client.0.smithi063.stderr: -246> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): _renew_subs 2024-05-09T20:28:02.054 INFO:tasks.workunit.client.0.smithi063.stderr: -245> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): authenticate will time out at 2089.073334s 2024-05-09T20:28:02.054 INFO:tasks.workunit.client.0.smithi063.stderr: -244> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): get_auth_request con 0x55e02eaf92d0 auth_method 0 2024-05-09T20:28:02.054 INFO:tasks.workunit.client.0.smithi063.stderr: -243> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.054 INFO:tasks.workunit.client.0.smithi063.stderr: -242> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.054 INFO:tasks.workunit.client.0.smithi063.stderr: -241> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.054 INFO:tasks.workunit.client.0.smithi063.stderr: -240> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -239> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -238> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -237> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient(hunting): get_auth_request con 0x55e02eb22860 auth_method 0 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -236> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -235> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -234> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -233> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): handle_auth_done global_id 6581 payload 274 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -232> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.055 INFO:tasks.workunit.client.0.smithi063.stderr: -231> 2024-05-09T20:28:01.932+0000 7f67f3de1640 1 monclient: found mon.noname-c 2024-05-09T20:28:02.056 INFO:tasks.workunit.client.0.smithi063.stderr: -230> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.056 INFO:tasks.workunit.client.0.smithi063.stderr: -229> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.056 INFO:tasks.workunit.client.0.smithi063.stderr: -228> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:28:02.056 INFO:tasks.workunit.client.0.smithi063.stderr: -227> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: dump: 2024-05-09T20:28:02.056 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.056 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.056 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.057 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.057 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.057 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.057 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.057 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.057 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.057 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.057 INFO:tasks.workunit.client.0.smithi063.stderr: -226> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.058 INFO:tasks.workunit.client.0.smithi063.stderr: -225> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.058 INFO:tasks.workunit.client.0.smithi063.stderr: -224> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.058 INFO:tasks.workunit.client.0.smithi063.stderr: -223> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.058 INFO:tasks.workunit.client.0.smithi063.stderr: -222> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:28:02.058 INFO:tasks.workunit.client.0.smithi063.stderr: -221> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: dump: 2024-05-09T20:28:02.058 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.058 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.058 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.059 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.059 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.059 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.059 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.059 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.059 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.060 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.060 INFO:tasks.workunit.client.0.smithi063.stderr: -220> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6581 2024-05-09T20:28:02.060 INFO:tasks.workunit.client.0.smithi063.stderr: -219> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: get_monmap_and_config success 2024-05-09T20:28:02.060 INFO:tasks.workunit.client.0.smithi063.stderr: -218> 2024-05-09T20:28:01.932+0000 7f67f62cd980 4 set_mon_vals no callback set 2024-05-09T20:28:02.060 INFO:tasks.workunit.client.0.smithi063.stderr: -217> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: shutdown 2024-05-09T20:28:02.061 INFO:tasks.workunit.client.0.smithi063.stderr: -216> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: build_initial_monmap 2024-05-09T20:28:02.061 INFO:tasks.workunit.client.0.smithi063.stderr: -215> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 build_initial for_mkfs: 0 2024-05-09T20:28:02.061 INFO:tasks.workunit.client.0.smithi063.stderr: -214> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: monmap: 2024-05-09T20:28:02.061 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:28:02.061 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:28:02.062 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:28:02.062 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:28:02.062 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:28:02.062 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.062 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:28:02.062 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -213> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -212> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -211> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -210> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -209> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -208> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -207> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -206> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -205> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.063 INFO:tasks.workunit.client.0.smithi063.stderr: -204> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -203> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -202> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -201> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -200> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -199> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -198> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -197> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 librados: starting msgr at 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -196> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 librados: starting objecter 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -195> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command objecter_requests cmddesc objecter_requests hook 0x55e02ea80320 EEXIST 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -194> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 librados: setting wanted keys 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -193> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 librados: calling monclient init 2024-05-09T20:28:02.064 INFO:tasks.workunit.client.0.smithi063.stderr: -192> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: init 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -191> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding auth protocol: cephx 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -190> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding auth protocol: cephx 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -189> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding auth protocol: cephx 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -188> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -187> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -186> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -185> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -184> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -183> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -182> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -181> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -180> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.065 INFO:tasks.workunit.client.0.smithi063.stderr: -179> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -178> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -177> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -176> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -175> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -174> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command rotate-key cmddesc rotate-key hook 0x55e02eaf3028 EEXIST 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -173> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: _reopen_session rank -1 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -172> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -171> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-c con 0x55e02eaf92d0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -170> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-a con 0x55e02eb22860 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -169> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-b con 0x55e02eb28250 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -168> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.066 INFO:tasks.workunit.client.0.smithi063.stderr: -167> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -166> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -165> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): _renew_subs 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -164> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): authenticate will time out at 2089.075938s 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -163> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): get_auth_request con 0x55e02eaf92d0 auth_method 0 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -162> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -161> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -160> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -159> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): get_auth_request con 0x55e02eb22860 auth_method 0 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -158> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.067 INFO:tasks.workunit.client.0.smithi063.stderr: -157> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -156> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -155> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -154> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -153> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -152> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -151> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -150> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -149> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): handle_auth_done global_id 6978 payload 824 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -148> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -147> 2024-05-09T20:28:01.936+0000 7f67f45e2640 1 monclient: found mon.noname-a 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -146> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -145> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.068 INFO:tasks.workunit.client.0.smithi063.stderr: -144> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr: -143> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: dump: 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.069 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr: -142> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr: -141> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr: -140> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr: -139> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr: -138> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr: -137> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: dump: 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.070 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: -136> 2024-05-09T20:28:01.936+0000 7f66d2ffd640 4 set_mon_vals no callback set 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: -135> 2024-05-09T20:28:01.936+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6978 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: -134> 2024-05-09T20:28:01.936+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: -133> 2024-05-09T20:28:01.936+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: -132> 2024-05-09T20:28:01.936+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: -131> 2024-05-09T20:28:01.936+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: -130> 2024-05-09T20:28:01.936+0000 7f67f62cd980 1 librados: init done 2024-05-09T20:28:02.071 INFO:tasks.workunit.client.0.smithi063.stderr: -129> 2024-05-09T20:28:01.936+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cr dump hook 0x55e02eb17c78 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -128> 2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -127> 2024-05-09T20:28:01.936+0000 7f66d17fa640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -126> 2024-05-09T20:28:01.936+0000 7f66d17fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -125> 2024-05-09T20:28:01.936+0000 7f66d17fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -124> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f66bc066d60 auth_method 0 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -123> 2024-05-09T20:28:01.936+0000 7f67f3de1640 10 monclient: get_auth_request con 0x55e02eb422d0 auth_method 0 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -122> 2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -121> 2024-05-09T20:28:01.936+0000 7f67f62cd980 20 realm 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -120> 2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.072 INFO:tasks.workunit.client.0.smithi063.stderr: -119> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -118> 2024-05-09T20:28:01.940+0000 7f67f62cd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -117> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -116> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -115> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -114> 2024-05-09T20:28:01.940+0000 7f67f35e0640 10 monclient: get_auth_request con 0x55e02eb46630 auth_method 0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -113> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -112> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -111> 2024-05-09T20:28:01.940+0000 7f67f45e2640 10 monclient: get_auth_request con 0x55e02eb49d30 auth_method 0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -110> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -109> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 searching for the correct realm 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -108> 2024-05-09T20:28:01.940+0000 7f67f3de1640 10 monclient: get_auth_request con 0x55e02eb4ead0 auth_method 0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -107> 2024-05-09T20:28:01.944+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f67e4039940 auth_method 0 2024-05-09T20:28:02.073 INFO:tasks.workunit.client.0.smithi063.stderr: -106> 2024-05-09T20:28:01.944+0000 7f67f45e2640 10 monclient: get_auth_request con 0x7f67e8108040 auth_method 0 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -105> 2024-05-09T20:28:01.944+0000 7f67f45e2640 0 -- 172.21.15.63:0/2688707805 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f67e8108040 msgr2=0x7f67e80dc360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -104> 2024-05-09T20:28:01.944+0000 7f66d17fa640 1 client.6978.objecter ms_handle_reset 0x7f67e8108040 session 0x7f67e8114040 osd.2 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -103> 2024-05-09T20:28:01.944+0000 7f66d17fa640 10 monclient: _renew_subs 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -102> 2024-05-09T20:28:01.944+0000 7f66d17fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -101> 2024-05-09T20:28:01.944+0000 7f67f45e2640 10 monclient: get_auth_request con 0x7f66bc077490 auth_method 0 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -100> 2024-05-09T20:28:01.944+0000 7f67f3de1640 10 monclient: get_auth_request con 0x7f67ec07a070 auth_method 0 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -99> 2024-05-09T20:28:01.948+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f67e4049040 auth_method 0 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -98> 2024-05-09T20:28:01.952+0000 7f67f35e0640 0 -- 172.21.15.63:0/2688707805 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f67e4039940 msgr2=0x7f67e400f3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -97> 2024-05-09T20:28:01.952+0000 7f66d17fa640 1 client.6978.objecter ms_handle_reset 0x7f67e4039940 session 0x7f67e402cbe0 osd.7 2024-05-09T20:28:02.074 INFO:tasks.workunit.client.0.smithi063.stderr: -96> 2024-05-09T20:28:01.952+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f67e8108040 auth_method 0 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -95> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -94> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -93> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -92> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -91> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -90> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -89> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -88> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -87> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -86> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:02.075 INFO:tasks.workunit.client.0.smithi063.stderr: -85> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -84> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -83> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 zone default found 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -82> 2024-05-09T20:28:01.960+0000 7f67f62cd980 4 Realm: () 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -81> 2024-05-09T20:28:01.960+0000 7f67f62cd980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -80> 2024-05-09T20:28:01.960+0000 7f67f62cd980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -79> 2024-05-09T20:28:01.960+0000 7f67f62cd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -78> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 zonegroup default 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -77> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -76> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -75> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -74> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.076 INFO:tasks.workunit.client.0.smithi063.stderr: -73> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -72> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -71> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 started sync module instance, tier type = 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -70> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -69> 2024-05-09T20:28:01.972+0000 7f66d17fa640 1 client.6978.objecter ms_handle_reset 0x7f67e8108040 session 0x7f67e402cbe0 osd.7 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -68> 2024-05-09T20:28:01.972+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f67e4039940 auth_method 0 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -67> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=0 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -66> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=1 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -65> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=2 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -64> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=3 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -63> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=4 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -62> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=5 2024-05-09T20:28:02.077 INFO:tasks.workunit.client.0.smithi063.stderr: -61> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=6 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -60> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=7 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -59> 2024-05-09T20:28:01.980+0000 7f67f62cd980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -58> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding auth protocol: cephx 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -57> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding auth protocol: cephx 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -56> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding auth protocol: cephx 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -55> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -54> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -53> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -52> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -51> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.078 INFO:tasks.workunit.client.0.smithi063.stderr: -50> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -49> 2024-05-09T20:28:01.984+0000 7f66aa7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -48> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -47> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -46> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -45> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -44> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -43> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -42> 2024-05-09T20:28:01.984+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -41> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -40> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -39> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cache list hook 0x55e02eb1e610 2024-05-09T20:28:02.079 INFO:tasks.workunit.client.0.smithi063.stderr: -38> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cache inspect hook 0x55e02eb1e610 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -37> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cache erase hook 0x55e02eb1e610 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -36> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cache zap hook 0x55e02eb1e610 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -35> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 note: GC not initialized 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -34> 2024-05-09T20:28:01.984+0000 7f66a97e2640 20 reqs_thread_entry: start 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -33> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command sync trace show hook 0x7f67e80043b0 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -32> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command sync trace history hook 0x7f67e80043b0 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -31> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command sync trace active hook 0x7f67e80043b0 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -30> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command sync trace active_short hook 0x7f67e80043b0 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -29> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -28> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 Filter name: none 2024-05-09T20:28:02.080 INFO:tasks.workunit.client.0.smithi063.stderr: -27> 2024-05-09T20:28:01.984+0000 7f66a27fc640 20 reqs_thread_entry: start 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -26> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -25> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -24> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -23> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -22> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -21> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 : miss 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -20> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -19> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -18> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -17> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -16> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:02.081 INFO:tasks.workunit.client.0.smithi063.stderr: -15> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -14> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -13> 2024-05-09T20:28:01.992+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -12> 2024-05-09T20:28:01.992+0000 7f67f62cd980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -11> 2024-05-09T20:28:01.992+0000 7f67f62cd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -10> 2024-05-09T20:28:01.992+0000 7f67f62cd980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28, ofs0, ns] 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -9> 2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431610 cookie 94421344665952 notifier 6978 bl.length()=903 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -8> 2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -7> 2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -6> 2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:02.082 INFO:tasks.workunit.client.0.smithi063.stderr: -5> 2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_sources= new_sources= 2024-05-09T20:28:02.083 INFO:tasks.workunit.client.0.smithi063.stderr: -4> 2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential sources added=[] removed=[] 2024-05-09T20:28:02.083 INFO:tasks.workunit.client.0.smithi063.stderr: -3> 2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_dests= new_dests= 2024-05-09T20:28:02.083 INFO:tasks.workunit.client.0.smithi063.stderr: -2> 2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential dests added=[] removed=[] 2024-05-09T20:28:02.083 INFO:tasks.workunit.client.0.smithi063.stderr: -1> 2024-05-09T20:28:02.004+0000 7f67f62cd980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f67f62cd980 time 2024-05-09T20:28:02.005601+0000 2024-05-09T20:28:02.083 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:28:02.083 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.083 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:28:02.083 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f67f9ba3e25] 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f67f9ba3fd9] 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e02c5b3ca1] 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31d) [0x55e02c5c441d] 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f67f8a0dd90] 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 0> 2024-05-09T20:28:02.004+0000 7f67f62cd980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f67f62cd980 thread_name:radosgw-admin 2024-05-09T20:28:02.084 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f67f8a26520] 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f67f9ba3e7f] 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f67f9ba3fd9] 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e02c5b3ca1] 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31d) [0x55e02c5c441d] 2024-05-09T20:28:02.085 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f67f8a0dd90] 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr:--- logging levels --- 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 none 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 lockdep 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 context 2024-05-09T20:28:02.086 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 crush 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_balancer 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_locker 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log_expire 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_migrator 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 3/ 5 mds_quiesce 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 buffer 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 timer 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 filer 2024-05-09T20:28:02.087 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 striper 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 objecter 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rados 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_mirror 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_replay 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_pwl 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 journaler 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objectcacher 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 immutable_obj_cache 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 client 2024-05-09T20:28:02.088 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 osd 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 optracker 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objclass 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 filestore 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 journal 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 ms 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mon 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 0/10 monc 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 paxos 2024-05-09T20:28:02.089 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 tp 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 auth 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 crypto 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 finisher 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 reserver 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 heartbeatmap 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 perfcounter 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 20/20 rgw 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_sync 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_datacache 2024-05-09T20:28:02.090 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_access 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_dbstore 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_flight 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_lifecycle 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_notification 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 javaclient 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 asok 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 throttle 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 refs 2024-05-09T20:28:02.091 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 compressor 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluestore 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluefs 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 bdev 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 kstore 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 4/ 5 rocksdb 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 fuse 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 2/ 5 mgr 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mgrc 2024-05-09T20:28:02.092 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 dpdk 2024-05-09T20:28:02.093 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 eventtrace 2024-05-09T20:28:02.093 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 prioritycache 2024-05-09T20:28:02.093 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 test 2024-05-09T20:28:02.093 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephfs_mirror 2024-05-09T20:28:02.093 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephsqlite 2024-05-09T20:28:02.093 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 crimson_interrupt 2024-05-09T20:28:02.093 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore 2024-05-09T20:28:02.093 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_onode 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_odata 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_omap 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_tm 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_t 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cleaner 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_epm 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_lba 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cache 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_journal 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_device 2024-05-09T20:28:02.094 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_backref 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 alienstore 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mclock 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cyanstore 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 ceph_exporter 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 memstore 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 trace 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 ceph_dedup 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: -2/-2 (syslog threshold) 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 99/99 (stderr threshold) 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66a27fc640 / http_manager 2024-05-09T20:28:02.095 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66a97e2640 / http_manager 2024-05-09T20:28:02.096 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66aa7e4640 / rgw_dt_lg_renew 2024-05-09T20:28:02.096 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66d17fa640 / ms_dispatch 2024-05-09T20:28:02.096 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66d2ffd640 / io_context_pool 2024-05-09T20:28:02.096 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66d97fa640 / ms_dispatch 2024-05-09T20:28:02.096 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66dbfff640 / ms_dispatch 2024-05-09T20:28:02.096 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66e1e0b640 / io_context_pool 2024-05-09T20:28:02.096 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f35e0640 / msgr-worker-2 2024-05-09T20:28:02.097 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f3de1640 / msgr-worker-1 2024-05-09T20:28:02.097 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f45e2640 / msgr-worker-0 2024-05-09T20:28:02.097 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f4de3640 / io_context_pool 2024-05-09T20:28:02.097 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f62cd980 / radosgw-admin 2024-05-09T20:28:02.097 INFO:tasks.workunit.client.0.smithi063.stderr: max_recent 500 2024-05-09T20:28:02.097 INFO:tasks.workunit.client.0.smithi063.stderr: max_new 1000 2024-05-09T20:28:02.097 INFO:tasks.workunit.client.0.smithi063.stderr: log_file /var/log/ceph/ceph-client.admin.100397.log 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr:--- end dump of recent events --- 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr:--- begin dump of recent events --- 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr: -499> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr: -498> 2024-05-09T20:28:01.916+0000 7f67f35e0640 1 monclient: found mon.noname-c 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr: -497> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr: -496> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr: -495> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr: -494> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: dump: 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.098 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr: -493> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr: -492> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr: -491> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr: -490> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr: -489> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:28:02.099 INFO:tasks.workunit.client.0.smithi063.stderr: -488> 2024-05-09T20:28:01.916+0000 7f66e1e0b640 10 monclient: dump: 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr: -487> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6572 2024-05-09T20:28:02.100 INFO:tasks.workunit.client.0.smithi063.stderr: -486> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: get_monmap_and_config success 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr: -485> 2024-05-09T20:28:01.916+0000 7f67f62cd980 4 set_mon_vals no callback set 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr: -484> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: shutdown 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr: -483> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 asok(0x55e02e84b430) unregister_commands rotate-key 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr: -482> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: build_initial_monmap 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr: -481> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 build_initial for_mkfs: 0 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr: -480> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: monmap: 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.101 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: -479> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding auth protocol: cephx 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: -478> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding auth protocol: cephx 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: -477> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding auth protocol: cephx 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: -476> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: -475> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: -474> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: -473> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.102 INFO:tasks.workunit.client.0.smithi063.stderr: -472> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -471> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -470> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -469> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -468> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -467> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -466> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: crc 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -465> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea5ab00) adding con mode: secure 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -464> 2024-05-09T20:28:01.916+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -463> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 librados: starting msgr at 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -462> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 librados: starting objecter 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -461> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command objecter_requests hook 0x55e02e8194c0 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -460> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 librados: setting wanted keys 2024-05-09T20:28:02.103 INFO:tasks.workunit.client.0.smithi063.stderr: -459> 2024-05-09T20:28:01.916+0000 7f67f62cd980 1 librados: calling monclient init 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -458> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: init 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -457> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding auth protocol: cephx 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -456> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding auth protocol: cephx 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -455> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding auth protocol: cephx 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -454> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -453> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -452> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -451> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -450> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -449> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.104 INFO:tasks.workunit.client.0.smithi063.stderr: -448> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -447> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -446> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -445> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -444> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: crc 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -443> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 AuthRegistry(0x55e02ea54ac0) adding con mode: secure 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -442> 2024-05-09T20:28:01.916+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -441> 2024-05-09T20:28:01.916+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -440> 2024-05-09T20:28:01.916+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command rotate-key hook 0x55e02ea54c08 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -439> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: _reopen_session rank -1 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -438> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient: _add_conns ranks=[1,0,2] 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -437> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-b con 0x55e02e906360 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:28:02.105 INFO:tasks.workunit.client.0.smithi063.stderr: -436> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-a con 0x55e02e910d70 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -435> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-c con 0x55e02e99bb40 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -434> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -433> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -432> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -431> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): _renew_subs 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -430> 2024-05-09T20:28:01.916+0000 7f67f62cd980 10 monclient(hunting): authenticate will time out at 2089.059621s 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -429> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): get_auth_request con 0x55e02e910d70 auth_method 0 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -428> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -427> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -426> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -425> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): get_auth_request con 0x55e02e99bb40 auth_method 0 2024-05-09T20:28:02.106 INFO:tasks.workunit.client.0.smithi063.stderr: -424> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -423> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -422> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -421> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -420> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -419> 2024-05-09T20:28:01.916+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -418> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -417> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -416> 2024-05-09T20:28:01.916+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -415> 2024-05-09T20:28:01.916+0000 7f67f3de1640 10 monclient(hunting): get_auth_request con 0x55e02e906360 auth_method 0 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -414> 2024-05-09T20:28:01.916+0000 7f67f3de1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -413> 2024-05-09T20:28:01.916+0000 7f67f3de1640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.107 INFO:tasks.workunit.client.0.smithi063.stderr: -412> 2024-05-09T20:28:01.916+0000 7f67f3de1640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr: -411> 2024-05-09T20:28:01.920+0000 7f67f45e2640 10 monclient(hunting): handle_auth_done global_id 6575 payload 824 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr: -410> 2024-05-09T20:28:01.920+0000 7f67f45e2640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr: -409> 2024-05-09T20:28:01.920+0000 7f67f45e2640 1 monclient: found mon.noname-c 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr: -408> 2024-05-09T20:28:01.920+0000 7f67f45e2640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr: -407> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr: -406> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr: -405> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: dump: 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.108 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr: -404> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr: -403> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr: -402> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr: -401> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr: -400> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr: -399> 2024-05-09T20:28:01.920+0000 7f66dbfff640 10 monclient: dump: 2024-05-09T20:28:02.109 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr: -398> 2024-05-09T20:28:01.920+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6575 2024-05-09T20:28:02.110 INFO:tasks.workunit.client.0.smithi063.stderr: -397> 2024-05-09T20:28:01.920+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -396> 2024-05-09T20:28:01.920+0000 7f66e1e0b640 4 set_mon_vals no callback set 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -395> 2024-05-09T20:28:01.920+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -394> 2024-05-09T20:28:01.920+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -393> 2024-05-09T20:28:01.920+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -392> 2024-05-09T20:28:01.920+0000 7f67f62cd980 1 librados: init done 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -391> 2024-05-09T20:28:01.920+0000 7f66dbfff640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -390> 2024-05-09T20:28:01.920+0000 7f66dbfff640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -389> 2024-05-09T20:28:01.920+0000 7f66dbfff640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -388> 2024-05-09T20:28:01.920+0000 7f67f3de1640 10 monclient: get_auth_request con 0x7f66dc066d60 auth_method 0 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -387> 2024-05-09T20:28:01.920+0000 7f67f35e0640 10 monclient: get_auth_request con 0x55e02ea7a1f0 auth_method 0 2024-05-09T20:28:02.111 INFO:tasks.workunit.client.0.smithi063.stderr: -386> 2024-05-09T20:28:01.924+0000 7f67f3de1640 10 monclient: get_auth_request con 0x55e02ea9fce0 auth_method 0 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr: -385> 2024-05-09T20:28:01.924+0000 7f67f35e0640 10 monclient: get_auth_request con 0x55e02eaa3ed0 auth_method 0 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr: -384> 2024-05-09T20:28:01.924+0000 7f67f45e2640 10 monclient: get_auth_request con 0x55e02eaa35c0 auth_method 0 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr: -383> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: build_initial_monmap 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr: -382> 2024-05-09T20:28:01.928+0000 7f67f62cd980 1 build_initial for_mkfs: 0 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr: -381> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: monmap: 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.112 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: -380> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding auth protocol: cephx 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: -379> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding auth protocol: cephx 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: -378> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding auth protocol: cephx 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: -377> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: -376> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: -375> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: -374> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.113 INFO:tasks.workunit.client.0.smithi063.stderr: -373> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -372> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -371> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -370> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -369> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -368> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -367> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: crc 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -366> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaabff0) adding con mode: secure 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -365> 2024-05-09T20:28:01.928+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -364> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command objecter_requests cmddesc objecter_requests hook 0x55e02eab1560 EEXIST 2024-05-09T20:28:02.114 INFO:tasks.workunit.client.0.smithi063.stderr: -363> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: init 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -362> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding auth protocol: cephx 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -361> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding auth protocol: cephx 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -360> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding auth protocol: cephx 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -359> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -358> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -357> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -356> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -355> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -354> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -353> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.115 INFO:tasks.workunit.client.0.smithi063.stderr: -352> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -351> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -350> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -349> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: crc 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -348> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaaaf08) adding con mode: secure 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -347> 2024-05-09T20:28:01.928+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -346> 2024-05-09T20:28:01.928+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -345> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command rotate-key cmddesc rotate-key hook 0x55e02eaab050 EEXIST 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -344> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _reopen_session rank -1 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -343> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -342> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-c con 0x55e02ead61c0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -341> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-a con 0x55e02ead65a0 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:28:02.116 INFO:tasks.workunit.client.0.smithi063.stderr: -340> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-b con 0x55e02ead1760 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -339> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -338> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -337> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -336> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): _renew_subs 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -335> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient(hunting): authenticate will time out at 2089.069228s 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -334> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): get_auth_request con 0x55e02ead61c0 auth_method 0 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -333> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -332> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -331> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -330> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): get_auth_request con 0x55e02ead65a0 auth_method 0 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -329> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.117 INFO:tasks.workunit.client.0.smithi063.stderr: -328> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -327> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -326> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -325> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -324> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -323> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -322> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -321> 2024-05-09T20:28:01.928+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -320> 2024-05-09T20:28:01.928+0000 7f67f3de1640 10 monclient(hunting): get_auth_request con 0x55e02ead1760 auth_method 0 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -319> 2024-05-09T20:28:01.928+0000 7f67f3de1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -318> 2024-05-09T20:28:01.928+0000 7f67f3de1640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -317> 2024-05-09T20:28:01.928+0000 7f67f3de1640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.118 INFO:tasks.workunit.client.0.smithi063.stderr: -316> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient(hunting): handle_auth_done global_id 6578 payload 824 2024-05-09T20:28:02.119 INFO:tasks.workunit.client.0.smithi063.stderr: -315> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.119 INFO:tasks.workunit.client.0.smithi063.stderr: -314> 2024-05-09T20:28:01.928+0000 7f67f35e0640 1 monclient: found mon.noname-c 2024-05-09T20:28:02.119 INFO:tasks.workunit.client.0.smithi063.stderr: -313> 2024-05-09T20:28:01.928+0000 7f67f35e0640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.119 INFO:tasks.workunit.client.0.smithi063.stderr: -312> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.119 INFO:tasks.workunit.client.0.smithi063.stderr: -311> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:28:02.119 INFO:tasks.workunit.client.0.smithi063.stderr: -310> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: dump: 2024-05-09T20:28:02.119 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr: -309> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr: -308> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr: -307> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.120 INFO:tasks.workunit.client.0.smithi063.stderr: -306> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr: -305> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr: -304> 2024-05-09T20:28:01.928+0000 7f66d97fa640 10 monclient: dump: 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.121 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr: -303> 2024-05-09T20:28:01.928+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6578 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr: -302> 2024-05-09T20:28:01.928+0000 7f67f4de3640 4 set_mon_vals no callback set 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr: -301> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr: -300> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr: -299> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr: -298> 2024-05-09T20:28:01.928+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.122 INFO:tasks.workunit.client.0.smithi063.stderr: -297> 2024-05-09T20:28:01.928+0000 7f66d97fa640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr: -296> 2024-05-09T20:28:01.928+0000 7f66d97fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr: -295> 2024-05-09T20:28:01.928+0000 7f66d97fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr: -294> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient: get_auth_request con 0x7f66cc066ca0 auth_method 0 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr: -293> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: get_monmap_and_config 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr: -292> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: build_initial_monmap 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr: -291> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 build_initial for_mkfs: 0 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr: -290> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: monmap: 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:28:02.123 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr: -289> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr: -288> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.124 INFO:tasks.workunit.client.0.smithi063.stderr: -287> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.125 INFO:tasks.workunit.client.0.smithi063.stderr: -286> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.125 INFO:tasks.workunit.client.0.smithi063.stderr: -285> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.125 INFO:tasks.workunit.client.0.smithi063.stderr: -284> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.125 INFO:tasks.workunit.client.0.smithi063.stderr: -283> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.125 INFO:tasks.workunit.client.0.smithi063.stderr: -282> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.125 INFO:tasks.workunit.client.0.smithi063.stderr: -281> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.125 INFO:tasks.workunit.client.0.smithi063.stderr: -280> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -279> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -278> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -277> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -276> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -275> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -274> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -273> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: init 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -272> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding auth protocol: cephx 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -271> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding auth protocol: cephx 2024-05-09T20:28:02.126 INFO:tasks.workunit.client.0.smithi063.stderr: -270> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding auth protocol: cephx 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -269> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -268> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -267> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -266> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -265> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -264> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -263> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -262> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -261> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.127 INFO:tasks.workunit.client.0.smithi063.stderr: -260> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.128 INFO:tasks.workunit.client.0.smithi063.stderr: -259> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: crc 2024-05-09T20:28:02.128 INFO:tasks.workunit.client.0.smithi063.stderr: -258> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebb9d0) adding con mode: secure 2024-05-09T20:28:02.128 INFO:tasks.workunit.client.0.smithi063.stderr: -257> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.128 INFO:tasks.workunit.client.0.smithi063.stderr: -256> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.128 INFO:tasks.workunit.client.0.smithi063.stderr: -255> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command rotate-key cmddesc rotate-key hook 0x7ffde2ebbb18 EEXIST 2024-05-09T20:28:02.128 INFO:tasks.workunit.client.0.smithi063.stderr: -254> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: _reopen_session rank -1 2024-05-09T20:28:02.128 INFO:tasks.workunit.client.0.smithi063.stderr: -253> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: _add_conns ranks=[0,1,2] 2024-05-09T20:28:02.129 INFO:tasks.workunit.client.0.smithi063.stderr: -252> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-a con 0x55e02eb28250 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:28:02.129 INFO:tasks.workunit.client.0.smithi063.stderr: -251> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-b con 0x55e02eb22860 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:28:02.129 INFO:tasks.workunit.client.0.smithi063.stderr: -250> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-c con 0x55e02eaf92d0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:28:02.129 INFO:tasks.workunit.client.0.smithi063.stderr: -249> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.129 INFO:tasks.workunit.client.0.smithi063.stderr: -248> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.129 INFO:tasks.workunit.client.0.smithi063.stderr: -247> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.129 INFO:tasks.workunit.client.0.smithi063.stderr: -246> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): _renew_subs 2024-05-09T20:28:02.129 INFO:tasks.workunit.client.0.smithi063.stderr: -245> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): authenticate will time out at 2089.073334s 2024-05-09T20:28:02.130 INFO:tasks.workunit.client.0.smithi063.stderr: -244> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): get_auth_request con 0x55e02eaf92d0 auth_method 0 2024-05-09T20:28:02.130 INFO:tasks.workunit.client.0.smithi063.stderr: -243> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.130 INFO:tasks.workunit.client.0.smithi063.stderr: -242> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.130 INFO:tasks.workunit.client.0.smithi063.stderr: -241> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.130 INFO:tasks.workunit.client.0.smithi063.stderr: -240> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.130 INFO:tasks.workunit.client.0.smithi063.stderr: -239> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.130 INFO:tasks.workunit.client.0.smithi063.stderr: -238> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.131 INFO:tasks.workunit.client.0.smithi063.stderr: -237> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient(hunting): get_auth_request con 0x55e02eb22860 auth_method 0 2024-05-09T20:28:02.131 INFO:tasks.workunit.client.0.smithi063.stderr: -236> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.131 INFO:tasks.workunit.client.0.smithi063.stderr: -235> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.131 INFO:tasks.workunit.client.0.smithi063.stderr: -234> 2024-05-09T20:28:01.932+0000 7f67f45e2640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.131 INFO:tasks.workunit.client.0.smithi063.stderr: -233> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient(hunting): handle_auth_done global_id 6581 payload 274 2024-05-09T20:28:02.131 INFO:tasks.workunit.client.0.smithi063.stderr: -232> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.131 INFO:tasks.workunit.client.0.smithi063.stderr: -231> 2024-05-09T20:28:01.932+0000 7f67f3de1640 1 monclient: found mon.noname-c 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr: -230> 2024-05-09T20:28:01.932+0000 7f67f3de1640 10 monclient: _send_mon_message to mon.noname-c at v2:172.21.15.63:3301/0 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr: -229> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr: -228> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr: -227> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: dump: 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.132 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr: -226> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr: -225> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr: -224> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr: -223> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr: -222> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr: -221> 2024-05-09T20:28:01.932+0000 7f66d2ffd640 10 monclient: dump: 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.133 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr: -220> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6581 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr: -219> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: get_monmap_and_config success 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr: -218> 2024-05-09T20:28:01.932+0000 7f67f62cd980 4 set_mon_vals no callback set 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr: -217> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: shutdown 2024-05-09T20:28:02.134 INFO:tasks.workunit.client.0.smithi063.stderr: -216> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: build_initial_monmap 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr: -215> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 build_initial for_mkfs: 0 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr: -214> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: monmap: 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 0 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 0.000000 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:created 0.000000 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 0 (unknown) 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.noname-a 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.noname-b 2024-05-09T20:28:02.135 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.noname-c 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -213> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -212> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -211> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding auth protocol: cephx 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -210> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -209> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -208> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -207> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -206> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -205> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -204> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.136 INFO:tasks.workunit.client.0.smithi063.stderr: -203> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -202> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -201> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -200> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: crc 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -199> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf3d10) adding con mode: secure 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -198> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -197> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 librados: starting msgr at 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -196> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 librados: starting objecter 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -195> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command objecter_requests cmddesc objecter_requests hook 0x55e02ea80320 EEXIST 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -194> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 librados: setting wanted keys 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -193> 2024-05-09T20:28:01.932+0000 7f67f62cd980 1 librados: calling monclient init 2024-05-09T20:28:02.137 INFO:tasks.workunit.client.0.smithi063.stderr: -192> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: init 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -191> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding auth protocol: cephx 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -190> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding auth protocol: cephx 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -189> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding auth protocol: cephx 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -188> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -187> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -186> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -185> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -184> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -183> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -182> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -181> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.138 INFO:tasks.workunit.client.0.smithi063.stderr: -180> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -179> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -178> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: crc 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -177> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 AuthRegistry(0x55e02eaf2ee0) adding con mode: secure 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -176> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -175> 2024-05-09T20:28:01.932+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -174> 2024-05-09T20:28:01.932+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command rotate-key cmddesc rotate-key hook 0x55e02eaf3028 EEXIST 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -173> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: _reopen_session rank -1 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -172> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient: _add_conns ranks=[2,0,1] 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -171> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-c con 0x55e02eaf92d0 addr [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -170> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-a con 0x55e02eb22860 addr [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] 2024-05-09T20:28:02.139 INFO:tasks.workunit.client.0.smithi063.stderr: -169> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): picked mon.noname-b con 0x55e02eb28250 addr [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -168> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -167> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -166> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): start opening mon connection 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -165> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): _renew_subs 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -164> 2024-05-09T20:28:01.932+0000 7f67f62cd980 10 monclient(hunting): authenticate will time out at 2089.075938s 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -163> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): get_auth_request con 0x55e02eaf92d0 auth_method 0 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -162> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -161> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -160> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.140 INFO:tasks.workunit.client.0.smithi063.stderr: -159> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): get_auth_request con 0x55e02eb22860 auth_method 0 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -158> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1] 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -157> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): _init_auth method 2 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -156> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): _init_auth creating new auth 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -155> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -154> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -153> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -152> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload 9 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -151> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more payload_len 9 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -150> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -149> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient(hunting): handle_auth_done global_id 6978 payload 824 2024-05-09T20:28:02.141 INFO:tasks.workunit.client.0.smithi063.stderr: -148> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient: _finish_hunting 0 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr: -147> 2024-05-09T20:28:01.936+0000 7f67f45e2640 1 monclient: found mon.noname-a 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr: -146> 2024-05-09T20:28:01.936+0000 7f67f45e2640 10 monclient: _send_mon_message to mon.noname-a at v2:172.21.15.63:3300/0 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr: -145> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr: -144> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr: -143> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: dump: 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.142 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr: -142> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: _finish_auth 0 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr: -141> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: _check_auth_tickets 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr: -140> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: handle_config config(0 keys) 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr: -139> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr: -138> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr: -137> 2024-05-09T20:28:01.936+0000 7f66d17fa640 10 monclient: dump: 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr:epoch 1 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr:fsid 9ed7a651-94d4-45eb-abce-d57b88f09598 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr:last_changed 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.143 INFO:tasks.workunit.client.0.smithi063.stderr:created 2024-05-09T20:08:46.704231+0000 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr:min_mon_release 19 (squid) 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr:election_strategy: 1 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr:0: [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon.a 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr:2: [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] mon.c 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr: -136> 2024-05-09T20:28:01.936+0000 7f66d2ffd640 4 set_mon_vals no callback set 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr: -135> 2024-05-09T20:28:01.936+0000 7f67f62cd980 5 monclient: authenticate success, global_id 6978 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr: -134> 2024-05-09T20:28:01.936+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr: -133> 2024-05-09T20:28:01.936+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr: -132> 2024-05-09T20:28:01.936+0000 7f67f62cd980 10 monclient: _renew_subs 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr: -131> 2024-05-09T20:28:01.936+0000 7f67f62cd980 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:28:02.144 INFO:tasks.workunit.client.0.smithi063.stderr: -130> 2024-05-09T20:28:01.936+0000 7f67f62cd980 1 librados: init done 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -129> 2024-05-09T20:28:01.936+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cr dump hook 0x55e02eb17c78 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -128> 2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -127> 2024-05-09T20:28:01.936+0000 7f66d17fa640 4 mgrc handle_mgr_map Got map version 5 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -126> 2024-05-09T20:28:01.936+0000 7f66d17fa640 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -125> 2024-05-09T20:28:01.936+0000 7f66d17fa640 4 mgrc reconnect Starting new session with [v2:172.21.15.63:6832/2991883022,v1:172.21.15.63:6833/2991883022] 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -124> 2024-05-09T20:28:01.936+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f66bc066d60 auth_method 0 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -123> 2024-05-09T20:28:01.936+0000 7f67f3de1640 10 monclient: get_auth_request con 0x55e02eb422d0 auth_method 0 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -122> 2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -121> 2024-05-09T20:28:01.936+0000 7f67f62cd980 20 realm 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -120> 2024-05-09T20:28:01.936+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.145 INFO:tasks.workunit.client.0.smithi063.stderr: -119> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -118> 2024-05-09T20:28:01.940+0000 7f67f62cd980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -117> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -116> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -115> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -114> 2024-05-09T20:28:01.940+0000 7f67f35e0640 10 monclient: get_auth_request con 0x55e02eb46630 auth_method 0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -113> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -112> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -111> 2024-05-09T20:28:01.940+0000 7f67f45e2640 10 monclient: get_auth_request con 0x55e02eb49d30 auth_method 0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -110> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -109> 2024-05-09T20:28:01.940+0000 7f67f62cd980 20 searching for the correct realm 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -108> 2024-05-09T20:28:01.940+0000 7f67f3de1640 10 monclient: get_auth_request con 0x55e02eb4ead0 auth_method 0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -107> 2024-05-09T20:28:01.944+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f67e4039940 auth_method 0 2024-05-09T20:28:02.146 INFO:tasks.workunit.client.0.smithi063.stderr: -106> 2024-05-09T20:28:01.944+0000 7f67f45e2640 10 monclient: get_auth_request con 0x7f67e8108040 auth_method 0 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -105> 2024-05-09T20:28:01.944+0000 7f67f45e2640 0 -- 172.21.15.63:0/2688707805 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f67e8108040 msgr2=0x7f67e80dc360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -104> 2024-05-09T20:28:01.944+0000 7f66d17fa640 1 client.6978.objecter ms_handle_reset 0x7f67e8108040 session 0x7f67e8114040 osd.2 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -103> 2024-05-09T20:28:01.944+0000 7f66d17fa640 10 monclient: _renew_subs 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -102> 2024-05-09T20:28:01.944+0000 7f66d17fa640 10 monclient: _send_mon_message to mon.a at v2:172.21.15.63:3300/0 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -101> 2024-05-09T20:28:01.944+0000 7f67f45e2640 10 monclient: get_auth_request con 0x7f66bc077490 auth_method 0 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -100> 2024-05-09T20:28:01.944+0000 7f67f3de1640 10 monclient: get_auth_request con 0x7f67ec07a070 auth_method 0 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -99> 2024-05-09T20:28:01.948+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f67e4049040 auth_method 0 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -98> 2024-05-09T20:28:01.952+0000 7f67f35e0640 0 -- 172.21.15.63:0/2688707805 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f67e4039940 msgr2=0x7f67e400f3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -97> 2024-05-09T20:28:01.952+0000 7f66d17fa640 1 client.6978.objecter ms_handle_reset 0x7f67e4039940 session 0x7f67e402cbe0 osd.7 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -96> 2024-05-09T20:28:01.952+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f67e8108040 auth_method 0 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -95> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -94> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:02.147 INFO:tasks.workunit.client.0.smithi063.stderr: -93> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -92> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -91> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -90> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -89> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -88> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -87> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -86> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -85> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.148 INFO:tasks.workunit.client.0.smithi063.stderr: -84> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -83> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 zone default found 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -82> 2024-05-09T20:28:01.960+0000 7f67f62cd980 4 Realm: () 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -81> 2024-05-09T20:28:01.960+0000 7f67f62cd980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -80> 2024-05-09T20:28:01.960+0000 7f67f62cd980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -79> 2024-05-09T20:28:01.960+0000 7f67f62cd980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -78> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 zonegroup default 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -77> 2024-05-09T20:28:01.960+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -76> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -75> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -74> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -73> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -72> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.149 INFO:tasks.workunit.client.0.smithi063.stderr: -71> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 started sync module instance, tier type = 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -70> 2024-05-09T20:28:01.964+0000 7f67f62cd980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -69> 2024-05-09T20:28:01.972+0000 7f66d17fa640 1 client.6978.objecter ms_handle_reset 0x7f67e8108040 session 0x7f67e402cbe0 osd.7 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -68> 2024-05-09T20:28:01.972+0000 7f67f35e0640 10 monclient: get_auth_request con 0x7f67e4039940 auth_method 0 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -67> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=0 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -66> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=1 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -65> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=2 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -64> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=3 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -63> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=4 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -62> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=5 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -61> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=6 2024-05-09T20:28:02.150 INFO:tasks.workunit.client.0.smithi063.stderr: -60> 2024-05-09T20:28:01.980+0000 7f67f62cd980 20 add_watcher() i=7 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -59> 2024-05-09T20:28:01.980+0000 7f67f62cd980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -58> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding auth protocol: cephx 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -57> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding auth protocol: cephx 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -56> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding auth protocol: cephx 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -55> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -54> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -53> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -52> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -51> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -50> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -49> 2024-05-09T20:28:01.984+0000 7f66aa7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:02.151 INFO:tasks.workunit.client.0.smithi063.stderr: -48> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -47> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -46> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -45> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -44> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: crc 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -43> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 AuthRegistry(0x7ffde2ebbe60) adding con mode: secure 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -42> 2024-05-09T20:28:01.984+0000 7f67f62cd980 2 auth: KeyRing::load: loaded key file /etc/ceph/ceph.keyring 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -41> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -40> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -39> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cache list hook 0x55e02eb1e610 2024-05-09T20:28:02.152 INFO:tasks.workunit.client.0.smithi063.stderr: -38> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cache inspect hook 0x55e02eb1e610 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -37> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cache erase hook 0x55e02eb1e610 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -36> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command cache zap hook 0x55e02eb1e610 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -35> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 note: GC not initialized 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -34> 2024-05-09T20:28:01.984+0000 7f66a97e2640 20 reqs_thread_entry: start 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -33> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command sync trace show hook 0x7f67e80043b0 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -32> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command sync trace history hook 0x7f67e80043b0 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -31> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command sync trace active hook 0x7f67e80043b0 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -30> 2024-05-09T20:28:01.984+0000 7f67f62cd980 5 asok(0x55e02e84b430) register_command sync trace active_short hook 0x7f67e80043b0 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -29> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:02.153 INFO:tasks.workunit.client.0.smithi063.stderr: -28> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 Filter name: none 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -27> 2024-05-09T20:28:01.984+0000 7f66a27fc640 20 reqs_thread_entry: start 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -26> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -25> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -24> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -23> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -22> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -21> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 : miss 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -20> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -19> 2024-05-09T20:28:01.984+0000 7f67f62cd980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -18> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:02.154 INFO:tasks.workunit.client.0.smithi063.stderr: -17> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:02.155 INFO:tasks.workunit.client.0.smithi063.stderr: -16> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:02.155 INFO:tasks.workunit.client.0.smithi063.stderr: -15> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:02.155 INFO:tasks.workunit.client.0.smithi063.stderr: -14> 2024-05-09T20:28:01.984+0000 7f67f62cd980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 2024-05-09T20:28:02.155 INFO:tasks.workunit.client.0.smithi063.stderr: -13> 2024-05-09T20:28:01.992+0000 7f67f62cd980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:02.155 INFO:tasks.workunit.client.0.smithi063.stderr: -12> 2024-05-09T20:28:01.992+0000 7f67f62cd980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:02.155 INFO:tasks.workunit.client.0.smithi063.stderr: -11> 2024-05-09T20:28:01.992+0000 7f67f62cd980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:02.155 INFO:tasks.workunit.client.0.smithi063.stderr: -10> 2024-05-09T20:28:01.992+0000 7f67f62cd980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28, ofs0, ns] 2024-05-09T20:28:02.155 INFO:tasks.workunit.client.0.smithi063.stderr: -9> 2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431610 cookie 94421344665952 notifier 6978 bl.length()=903 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: -8> 2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: -7> 2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: -6> 2024-05-09T20:28:01.996+0000 7f66d2ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: -5> 2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_sources= new_sources= 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: -4> 2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential sources added=[] removed=[] 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: -3> 2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_dests= new_dests= 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: -2> 2024-05-09T20:28:01.996+0000 7f67f62cd980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential dests added=[] removed=[] 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: -1> 2024-05-09T20:28:02.004+0000 7f67f62cd980 -1 ./src/common/fault_injector.h: In function 'int FaultInjector::check(const Key&) const::visitor::operator()(const InjectAbort&) const [with Key = std::basic_string_view]' thread 7f67f62cd980 time 2024-05-09T20:28:02.005601+0000 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr:./src/common/fault_injector.h: 120: FAILED ceph_assert(!"FaultInjector") 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:28:02.156 INFO:tasks.workunit.client.0.smithi063.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x121) [0x7f67f9ba3e25] 2024-05-09T20:28:02.157 INFO:tasks.workunit.client.0.smithi063.stderr: 2: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f67f9ba3fd9] 2024-05-09T20:28:02.157 INFO:tasks.workunit.client.0.smithi063.stderr: 3: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e02c5b3ca1] 2024-05-09T20:28:02.157 INFO:tasks.workunit.client.0.smithi063.stderr: 4: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31d) [0x55e02c5c441d] 2024-05-09T20:28:02.157 INFO:tasks.workunit.client.0.smithi063.stderr: 5: main() 2024-05-09T20:28:02.157 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f67f8a0dd90] 2024-05-09T20:28:02.157 INFO:tasks.workunit.client.0.smithi063.stderr: 7: __libc_start_main() 2024-05-09T20:28:02.157 INFO:tasks.workunit.client.0.smithi063.stderr: 8: _start() 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 0> 2024-05-09T20:28:02.004+0000 7f67f62cd980 -1 *** Caught signal (Aborted) ** 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: in thread 7f67f62cd980 thread_name:radosgw-admin 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: ceph version 19.0.0-3307-g04969094 (04969094632d400ff0930fc029b1d19b9c187eef) squid (dev) 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 1: /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f67f8a26520] 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 2: pthread_kill() 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 3: raise() 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 4: abort() 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x17b) [0x7f67f9ba3e7f] 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 6: /usr/lib/ceph/libceph-common.so.2(+0x181fd9) [0x7f67f9ba3fd9] 2024-05-09T20:28:02.158 INFO:tasks.workunit.client.0.smithi063.stderr: 7: (std::__detail::__variant::__gen_vtable_impl (*)(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)>, std::integer_sequence >::__visit_invoke(FaultInjector > >::check(std::basic_string_view > const&) const::visitor&&, std::variant const&)+0x41) [0x55e02c5b3ca1] 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr: 8: (RGWBucketReshard::execute(int, FaultInjector > >&, int, DoutPrefixProvider const*, optional_yield, bool, std::ostream*, ceph::Formatter*, RGWReshard*)+0x31d) [0x55e02c5c441d] 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr: 9: main() 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr: 10: /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f67f8a0dd90] 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr: 11: __libc_start_main() 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr: 12: _start() 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr:--- logging levels --- 2024-05-09T20:28:02.159 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 none 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 lockdep 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 context 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 crush 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_balancer 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_locker 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_log_expire 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mds_migrator 2024-05-09T20:28:02.160 INFO:tasks.workunit.client.0.smithi063.stderr: 3/ 5 mds_quiesce 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 buffer 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 timer 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 filer 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 striper 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 1 objecter 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rados 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_mirror 2024-05-09T20:28:02.161 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_replay 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 rbd_pwl 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 journaler 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objectcacher 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 immutable_obj_cache 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 client 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 osd 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 optracker 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 objclass 2024-05-09T20:28:02.162 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 filestore 2024-05-09T20:28:02.163 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 journal 2024-05-09T20:28:02.163 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 ms 2024-05-09T20:28:02.163 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mon 2024-05-09T20:28:02.163 INFO:tasks.workunit.client.0.smithi063.stderr: 0/10 monc 2024-05-09T20:28:02.163 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 paxos 2024-05-09T20:28:02.163 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 tp 2024-05-09T20:28:02.163 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 auth 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 crypto 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 finisher 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 reserver 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 heartbeatmap 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 perfcounter 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 20/20 rgw 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_sync 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_datacache 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_access 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_dbstore 2024-05-09T20:28:02.164 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_flight 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_lifecycle 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 rgw_notification 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 javaclient 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 asok 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 1 throttle 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 0 refs 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 compressor 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluestore 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 bluefs 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 3 bdev 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 kstore 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 4/ 5 rocksdb 2024-05-09T20:28:02.165 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 fuse 2024-05-09T20:28:02.166 INFO:tasks.workunit.client.0.smithi063.stderr: 2/ 5 mgr 2024-05-09T20:28:02.166 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mgrc 2024-05-09T20:28:02.166 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 dpdk 2024-05-09T20:28:02.166 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 eventtrace 2024-05-09T20:28:02.166 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 prioritycache 2024-05-09T20:28:02.166 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 test 2024-05-09T20:28:02.166 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephfs_mirror 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cephsqlite 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 crimson_interrupt 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_onode 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_odata 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_omap 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_tm 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_t 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cleaner 2024-05-09T20:28:02.167 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_epm 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_lba 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_fixedkv_tree 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_cache 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_journal 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_device 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 seastore_backref 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 alienstore 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 mclock 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 cyanstore 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 ceph_exporter 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 memstore 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 1/ 5 trace 2024-05-09T20:28:02.168 INFO:tasks.workunit.client.0.smithi063.stderr: 0/ 5 ceph_dedup 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: -2/-2 (syslog threshold) 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 99/99 (stderr threshold) 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr:--- pthread ID / name mapping for recent threads --- 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66a27fc640 / http_manager 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66a97e2640 / http_manager 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66aa7e4640 / rgw_dt_lg_renew 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66d17fa640 / ms_dispatch 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66d2ffd640 / io_context_pool 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66d97fa640 / ms_dispatch 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66dbfff640 / ms_dispatch 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f66e1e0b640 / io_context_pool 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f35e0640 / msgr-worker-2 2024-05-09T20:28:02.169 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f3de1640 / msgr-worker-1 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f45e2640 / msgr-worker-0 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f4de3640 / io_context_pool 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr: 7f67f62cd980 / radosgw-admin 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr: max_recent 500 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr: max_new 1000 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr: log_file /var/lib/ceph/crash/2024-05-09T20:28:02.007574Z_8b8b2602-32c2-4a30-8d4a-cab485b294bb/log 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr:--- end dump of recent events --- 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr:Aborted 2024-05-09T20:28:02.170 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 20 realm 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:02.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.084+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.088+0000 7f26ceee1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:02.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.088+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.088+0000 7f26ceee1980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:02.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.088+0000 7f26ceee1980 20 searching for the correct realm 2024-05-09T20:28:02.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.092+0000 7f26cc9f5640 0 -- 172.21.15.63:0/3129725974 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f26c40e6b10 msgr2=0x7f26c40b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:02.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.304+0000 7f26cd1f6640 0 -- 172.21.15.63:0/3129725974 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x55ea9d4e0b10 msgr2=0x7f26c0076ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:02.311 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:02.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:02.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:02.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:02.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:02.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:02.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:02.313 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.320+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:02.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 zone default found 2024-05-09T20:28:02.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 4 Realm: () 2024-05-09T20:28:02.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:02.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:02.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:02.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 zonegroup default 2024-05-09T20:28:02.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:02.316 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 started sync module instance, tier type = 2024-05-09T20:28:02.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.324+0000 7f26ceee1980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:02.339 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 20 add_watcher() i=0 2024-05-09T20:28:02.339 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 20 add_watcher() i=1 2024-05-09T20:28:02.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 20 add_watcher() i=2 2024-05-09T20:28:02.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 20 add_watcher() i=3 2024-05-09T20:28:02.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 20 add_watcher() i=4 2024-05-09T20:28:02.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 20 add_watcher() i=5 2024-05-09T20:28:02.340 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 20 add_watcher() i=6 2024-05-09T20:28:02.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 20 add_watcher() i=7 2024-05-09T20:28:02.341 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.348+0000 7f26ceee1980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:02.344 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.352+0000 7f2582fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:02.345 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:02.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:02.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 5 note: GC not initialized 2024-05-09T20:28:02.346 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f2581fe3640 20 reqs_thread_entry: start 2024-05-09T20:28:02.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:02.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 Filter name: none 2024-05-09T20:28:02.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f24f37fe640 20 reqs_thread_entry: start 2024-05-09T20:28:02.347 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:02.348 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 rados_obj.operate() r=0 bl.length=433 2024-05-09T20:28:02.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:02.349 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.356+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.360+0000 7f26ceee1980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:02.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.360+0000 7f26ceee1980 20 rados->read ofs=0 len=0 2024-05-09T20:28:02.350 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.360+0000 7f26ceee1980 20 rados_obj.operate() r=0 bl.length=433 2024-05-09T20:28:02.351 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.360+0000 7f26ceee1980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:02.354 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.364+0000 7f2582fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:02.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.372+0000 7f26ceee1980 20 remove_watcher() i=0 2024-05-09T20:28:02.361 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.372+0000 7f26ceee1980 2 removed watcher, disabling cache 2024-05-09T20:28:02.364 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.372+0000 7f26ceee1980 20 remove_watcher() i=1 2024-05-09T20:28:02.366 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.376+0000 7f26ceee1980 20 remove_watcher() i=2 2024-05-09T20:28:02.369 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.376+0000 7f26ceee1980 20 remove_watcher() i=3 2024-05-09T20:28:02.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.380+0000 7f26ceee1980 20 remove_watcher() i=4 2024-05-09T20:28:02.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.384+0000 7f26ceee1980 20 remove_watcher() i=5 2024-05-09T20:28:02.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.384+0000 7f26ceee1980 20 remove_watcher() i=6 2024-05-09T20:28:02.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:02.384+0000 7f26ceee1980 20 remove_watcher() i=7 2024-05-09T20:28:02.393 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:02.394 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:28:02.394 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-do-reshard id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 num_objects: 20 size_kb: 1 num_shards: 11 2024-05-09T20:28:02.394 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:32.739 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket abort-at-do-reshard --num-shards 12 --rgw-reshard-bucket-lock-duration 30 2024-05-09T20:28:32.776 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:32.776 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:32.823 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.831+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:32.824 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.831+0000 7facaf939980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:32.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.831+0000 7facaf939980 20 realm 2024-05-09T20:28:32.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.831+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:32.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:32.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:32.825 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:32.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:32.826 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:32.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:32.827 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:32.828 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:32.828 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.835+0000 7facaf939980 20 searching for the correct realm 2024-05-09T20:28:32.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:32.839+0000 7facadc4e640 0 -- 172.21.15.63:0/3374692108 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fac98041030 msgr2=0x7fac980907b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:33.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.055+0000 7faca7fff640 0 -- 172.21.15.63:0/3374692108 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7faca0107770 msgr2=0x7faca00ac030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:33.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:33.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:33.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:33.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:33.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:33.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:33.064 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.071+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:33.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:33.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 zone default found 2024-05-09T20:28:33.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 4 Realm: () 2024-05-09T20:28:33.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:33.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:33.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:33.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 zonegroup default 2024-05-09T20:28:33.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 started sync module instance, tier type = 2024-05-09T20:28:33.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.075+0000 7facaf939980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 20 add_watcher() i=0 2024-05-09T20:28:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 20 add_watcher() i=1 2024-05-09T20:28:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 20 add_watcher() i=2 2024-05-09T20:28:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 20 add_watcher() i=3 2024-05-09T20:28:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 20 add_watcher() i=4 2024-05-09T20:28:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 20 add_watcher() i=5 2024-05-09T20:28:33.090 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 20 add_watcher() i=6 2024-05-09T20:28:33.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 20 add_watcher() i=7 2024-05-09T20:28:33.092 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.099+0000 7facaf939980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:33.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.103+0000 7fab637e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.103+0000 7facaf939980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 5 note: GC not initialized 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7fab627e4640 20 reqs_thread_entry: start 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 20 Filter name: none 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7faad3fff640 20 reqs_thread_entry: start 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 cache get: name=default.rgw.meta+root+abort-at-do-reshard : miss 2024-05-09T20:28:33.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 cache put: name=default.rgw.meta+root+abort-at-do-reshard info.flags=0x11 2024-05-09T20:28:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 adding default.rgw.meta+root+abort-at-do-reshard to cache LRU end 2024-05-09T20:28:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 : miss 2024-05-09T20:28:33.100 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:28:33.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:33.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 adding default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:33.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:33.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:33.101 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.107+0000 7facaf939980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 2024-05-09T20:28:33.109 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.119+0000 7facaf939980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:33.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.119+0000 7facaf939980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:33.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.119+0000 7facaf939980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:33.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.119+0000 7facaf939980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28, ofs0, ns] 2024-05-09T20:28:33.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.119+0000 7fab7ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431612 cookie 94201534708048 notifier 7014 bl.length()=903 2024-05-09T20:28:33.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.119+0000 7fab7ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:33.110 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.119+0000 7fab7ffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:33.111 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.119+0000 7fab7ffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:33.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.123+0000 7facaf939980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_sources= new_sources= 2024-05-09T20:28:33.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.123+0000 7facaf939980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential sources added=[] removed=[] 2024-05-09T20:28:33.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.123+0000 7facaf939980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_dests= new_dests= 2024-05-09T20:28:33.114 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.123+0000 7facaf939980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential dests added=[] removed=[] 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.0 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.0 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.1 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.1 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.2 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.2 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.3 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.3 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.4 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.4 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.5 2024-05-09T20:28:33.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.5 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.6 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.6 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.7 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.7 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.8 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.8 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.9 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.9 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.10 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.10 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.11 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1.11 2024-05-09T20:28:33.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.0 2024-05-09T20:28:33.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.127+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.1 2024-05-09T20:28:33.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.131+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.2 2024-05-09T20:28:33.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.131+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.3 2024-05-09T20:28:33.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.131+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.4 2024-05-09T20:28:33.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.131+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.5 2024-05-09T20:28:33.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.131+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.6 2024-05-09T20:28:33.126 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.135+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.7 2024-05-09T20:28:33.127 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.135+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.8 2024-05-09T20:28:33.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.135+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.9 2024-05-09T20:28:33.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.135+0000 7facaf939980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28.10 2024-05-09T20:28:33.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.143+0000 7facaf939980 10 cache get: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 : hit (requested=0x17, cached=0x17) 2024-05-09T20:28:33.135 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.143+0000 7facaf939980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 2024-05-09T20:28:33.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.147+0000 7facaf939980 10 cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:33.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.147+0000 7facaf939980 10 moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:33.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.147+0000 7facaf939980 10 updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:33.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.147+0000 7facaf939980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28, ofs0, ns] 2024-05-09T20:28:33.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.147+0000 7fab7ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431613 cookie 94201534708048 notifier 7014 bl.length()=871 2024-05-09T20:28:33.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.147+0000 7fab7ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 info.flags=0x17 2024-05-09T20:28:33.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.147+0000 7fab7ffff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.abort-at-do-reshard:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 to cache LRU end 2024-05-09T20:28:33.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.147+0000 7fab7ffff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=205 2024-05-09T20:28:33.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.151+0000 7facaf939980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_sources= new_sources= 2024-05-09T20:28:33.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.151+0000 7facaf939980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential sources added=[] removed=[] 2024-05-09T20:28:33.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.151+0000 7facaf939980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): orig_dests= new_dests= 2024-05-09T20:28:33.142 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.151+0000 7facaf939980 20 handle_bi_update(): bucket=:abort-at-do-reshard[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28]): potential dests added=[] removed=[] 2024-05-09T20:28:33.147 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.155+0000 7facaf939980 1 execute INFO: reshard of bucket "abort-at-do-reshard" completed successfully 2024-05-09T20:28:33.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.159+0000 7fab637e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:33.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.163+0000 7facaf939980 20 remove_watcher() i=0 2024-05-09T20:28:33.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.163+0000 7facaf939980 2 removed watcher, disabling cache 2024-05-09T20:28:33.158 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.167+0000 7facaf939980 20 remove_watcher() i=1 2024-05-09T20:28:33.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.167+0000 7facaf939980 20 remove_watcher() i=2 2024-05-09T20:28:33.163 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.171+0000 7facaf939980 20 remove_watcher() i=3 2024-05-09T20:28:33.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.171+0000 7facaf939980 20 remove_watcher() i=4 2024-05-09T20:28:33.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.175+0000 7facaf939980 20 remove_watcher() i=5 2024-05-09T20:28:33.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.175+0000 7facaf939980 20 remove_watcher() i=6 2024-05-09T20:28:33.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.179+0000 7facaf939980 20 remove_watcher() i=7 2024-05-09T20:28:33.188 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket abort-at-do-reshard 2024-05-09T20:28:33.217 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:33.218 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:33.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 20 realm 2024-05-09T20:28:33.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:33.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.271+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.275+0000 7f7088f5a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:33.265 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.275+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.275+0000 7f7088f5a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:33.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.275+0000 7f7088f5a980 20 searching for the correct realm 2024-05-09T20:28:33.269 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.279+0000 7f708726f640 0 -- 172.21.15.63:0/3090641191 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f707c0e3060 msgr2=0x7f707c122950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:33.484 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.491+0000 7f708626d640 0 -- 172.21.15.63:0/3090641191 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f7078023070 msgr2=0x7f707805ef20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.507+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:33.501 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.502 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 zone default found 2024-05-09T20:28:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 4 Realm: () 2024-05-09T20:28:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 zonegroup default 2024-05-09T20:28:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.503 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.511+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.515+0000 7f7088f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:33.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.515+0000 7f7088f5a980 20 started sync module instance, tier type = 2024-05-09T20:28:33.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.515+0000 7f7088f5a980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:33.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 20 add_watcher() i=0 2024-05-09T20:28:33.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 20 add_watcher() i=1 2024-05-09T20:28:33.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 20 add_watcher() i=2 2024-05-09T20:28:33.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 20 add_watcher() i=3 2024-05-09T20:28:33.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 20 add_watcher() i=4 2024-05-09T20:28:33.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 20 add_watcher() i=5 2024-05-09T20:28:33.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 20 add_watcher() i=6 2024-05-09T20:28:33.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 20 add_watcher() i=7 2024-05-09T20:28:33.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.539+0000 7f7088f5a980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:33.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.543+0000 7f6f3d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:33.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.543+0000 7f7088f5a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:33.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.543+0000 7f7088f5a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:33.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 5 note: GC not initialized 2024-05-09T20:28:33.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f6eb7fff640 20 reqs_thread_entry: start 2024-05-09T20:28:33.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:33.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 20 Filter name: none 2024-05-09T20:28:33.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f6eb5ffb640 20 reqs_thread_entry: start 2024-05-09T20:28:33.538 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:33.539 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:28:33.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:33.540 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.547+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.551+0000 7f7088f5a980 20 rados_obj.operate() r=0 bl.length=187 2024-05-09T20:28:33.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.551+0000 7f7088f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:28:33.541 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.551+0000 7f7088f5a980 20 rados_obj.operate() r=0 bl.length=401 2024-05-09T20:28:33.542 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.551+0000 7f7088f5a980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:33.545 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.555+0000 7f6f3d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:33.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.559+0000 7f7088f5a980 20 remove_watcher() i=0 2024-05-09T20:28:33.550 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.559+0000 7f7088f5a980 2 removed watcher, disabling cache 2024-05-09T20:28:33.553 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.563+0000 7f7088f5a980 20 remove_watcher() i=1 2024-05-09T20:28:33.555 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.563+0000 7f7088f5a980 20 remove_watcher() i=2 2024-05-09T20:28:33.557 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.567+0000 7f7088f5a980 20 remove_watcher() i=3 2024-05-09T20:28:33.559 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.567+0000 7f7088f5a980 20 remove_watcher() i=4 2024-05-09T20:28:33.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.571+0000 7f7088f5a980 20 remove_watcher() i=5 2024-05-09T20:28:33.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.571+0000 7f7088f5a980 20 remove_watcher() i=6 2024-05-09T20:28:33.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:33.571+0000 7f7088f5a980 20 remove_watcher() i=7 2024-05-09T20:28:33.582 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:33.582 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:28:33.582 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: abort-at-do-reshard id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.28 num_objects: 19 size_kb: 1 num_shards: 12 2024-05-09T20:28:33.582 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:34.494 INFO:tasks.workunit.client.0.smithi063.stderr: test: reshard versioned bucket 2024-05-09T20:28:34.494 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket myver 2024-05-09T20:28:34.527 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:34.527 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:34.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.579+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 realm 2024-05-09T20:28:34.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:34.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:34.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.583+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.587+0000 7f88c3449980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:34.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.587+0000 7f88c3449980 20 searching for the correct realm 2024-05-09T20:28:34.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.587+0000 7f88c175e640 0 -- 172.21.15.63:0/24255578 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f88a803c2a0 msgr2=0x7f88a8090a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:34.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.799+0000 7f88b875c640 0 -- 172.21.15.63:0/24255578 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f88b0033910 msgr2=0x7f88b00403f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:34.808 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:34.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:34.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:34.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:34.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:34.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:34.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.815+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:34.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:34.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 zone default found 2024-05-09T20:28:34.810 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 4 Realm: () 2024-05-09T20:28:34.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:34.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:34.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:34.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 zonegroup default 2024-05-09T20:28:34.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.811 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 started sync module instance, tier type = 2024-05-09T20:28:34.813 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.819+0000 7f88c3449980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:34.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.843+0000 7f88c3449980 20 add_watcher() i=0 2024-05-09T20:28:34.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.843+0000 7f88c3449980 20 add_watcher() i=1 2024-05-09T20:28:34.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.843+0000 7f88c3449980 20 add_watcher() i=2 2024-05-09T20:28:34.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.843+0000 7f88c3449980 20 add_watcher() i=3 2024-05-09T20:28:34.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.843+0000 7f88c3449980 20 add_watcher() i=4 2024-05-09T20:28:34.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.843+0000 7f88c3449980 20 add_watcher() i=5 2024-05-09T20:28:34.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.843+0000 7f88c3449980 20 add_watcher() i=6 2024-05-09T20:28:34.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.847+0000 7f88c3449980 20 add_watcher() i=7 2024-05-09T20:28:34.837 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.847+0000 7f88c3449980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:34.841 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f8776fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:34.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f88c3449980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:34.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f88c3449980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:34.842 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f88c3449980 5 note: GC not initialized 2024-05-09T20:28:34.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f8775fe3640 20 reqs_thread_entry: start 2024-05-09T20:28:34.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f88c3449980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:34.843 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f88c3449980 20 Filter name: none 2024-05-09T20:28:34.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f86e77fe640 20 reqs_thread_entry: start 2024-05-09T20:28:34.844 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f88c3449980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:34.845 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.851+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.855+0000 7f88c3449980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:34.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.855+0000 7f88c3449980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:34.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.855+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.855+0000 7f88c3449980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:34.846 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.855+0000 7f88c3449980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.855+0000 7f88c3449980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:34.847 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.855+0000 7f88c3449980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:34.851 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.859+0000 7f8776fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:34.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.863+0000 7f88c3449980 20 remove_watcher() i=0 2024-05-09T20:28:34.856 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.863+0000 7f88c3449980 2 removed watcher, disabling cache 2024-05-09T20:28:34.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.867+0000 7f88c3449980 20 remove_watcher() i=1 2024-05-09T20:28:34.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.871+0000 7f88c3449980 20 remove_watcher() i=2 2024-05-09T20:28:34.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.871+0000 7f88c3449980 20 remove_watcher() i=3 2024-05-09T20:28:34.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.875+0000 7f88c3449980 20 remove_watcher() i=4 2024-05-09T20:28:34.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.875+0000 7f88c3449980 20 remove_watcher() i=5 2024-05-09T20:28:34.869 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.875+0000 7f88c3449980 20 remove_watcher() i=6 2024-05-09T20:28:34.870 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.879+0000 7f88c3449980 20 remove_watcher() i=7 2024-05-09T20:28:34.888 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:34.888 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:28:34.888 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: myver id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 num_objects: 0 size_kb: 0 num_shards: 11 2024-05-09T20:28:34.888 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:34.888 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 12 2024-05-09T20:28:34.922 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:34.922 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:34.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.975+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 realm 2024-05-09T20:28:34.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:34.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:34.970 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:34.972 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:34.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:34.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.979+0000 7fcbf30df980 20 searching for the correct realm 2024-05-09T20:28:34.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:34.983+0000 7fcbf03f2640 0 -- 172.21.15.63:0/1739667297 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fcbe002f960 msgr2=0x7fcbe000d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:35.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.195+0000 7fcbf03f2640 0 -- 172.21.15.63:0/1739667297 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7fcbe0043900 msgr2=0x7fcbe002e3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:35.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:35.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:35.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:35.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:35.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:35.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:35.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:35.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:35.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 zone default found 2024-05-09T20:28:35.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 4 Realm: () 2024-05-09T20:28:35.412 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:35.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:35.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:35.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 zonegroup default 2024-05-09T20:28:35.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.419+0000 7fcbf30df980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.413 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.423+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.423+0000 7fcbf30df980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.423+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.423+0000 7fcbf30df980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.423+0000 7fcbf30df980 20 started sync module instance, tier type = 2024-05-09T20:28:35.414 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.423+0000 7fcbf30df980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:35.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 20 add_watcher() i=0 2024-05-09T20:28:35.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 20 add_watcher() i=1 2024-05-09T20:28:35.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 20 add_watcher() i=2 2024-05-09T20:28:35.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 20 add_watcher() i=3 2024-05-09T20:28:35.437 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 20 add_watcher() i=4 2024-05-09T20:28:35.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 20 add_watcher() i=5 2024-05-09T20:28:35.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 20 add_watcher() i=6 2024-05-09T20:28:35.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 20 add_watcher() i=7 2024-05-09T20:28:35.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.447+0000 7fcbf30df980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:35.443 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.451+0000 7fcaa77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:35.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.451+0000 7fcbf30df980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:35.444 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.451+0000 7fcbf30df980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:35.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.451+0000 7fcbf30df980 5 note: GC not initialized 2024-05-09T20:28:35.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcaa67e4640 20 reqs_thread_entry: start 2024-05-09T20:28:35.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:35.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 20 Filter name: none 2024-05-09T20:28:35.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fca9f7fe640 20 reqs_thread_entry: start 2024-05-09T20:28:35.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-05-09T20:28:35.446 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:35.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-05-09T20:28:35.447 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-05-09T20:28:35.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 : miss 2024-05-09T20:28:35.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:35.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:35.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 adding default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:35.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:35.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:35.448 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.455+0000 7fcbf30df980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 2024-05-09T20:28:35.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcbf30df980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:35.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcbf30df980 10 moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:35.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcbf30df980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:35.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcbf30df980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18, ofs0, ns] 2024-05-09T20:28:35.457 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcac7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431605 cookie 94464613232144 notifier 7074 bl.length()=813 2024-05-09T20:28:35.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcac7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:35.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcac7fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:35.458 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcac7fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:35.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcbf30df980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_sources= new_sources= 2024-05-09T20:28:35.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcbf30df980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential sources added=[] removed=[] 2024-05-09T20:28:35.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcbf30df980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_dests= new_dests= 2024-05-09T20:28:35.460 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.467+0000 7fcbf30df980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential dests added=[] removed=[] 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.0 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.0 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.1 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.1 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.2 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.2 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.3 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.3 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.4 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.4 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.5 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.5 2024-05-09T20:28:35.467 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.6 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.6 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.7 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.7 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.8 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.8 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.9 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.9 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.10 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.10 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.11 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.11 2024-05-09T20:28:35.468 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.0 2024-05-09T20:28:35.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.475+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1 2024-05-09T20:28:35.469 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.479+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2 2024-05-09T20:28:35.470 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.479+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3 2024-05-09T20:28:35.471 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.479+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.4 2024-05-09T20:28:35.472 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.479+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.5 2024-05-09T20:28:35.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.483+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.6 2024-05-09T20:28:35.474 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.483+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.7 2024-05-09T20:28:35.475 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.483+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.8 2024-05-09T20:28:35.476 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.483+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.9 2024-05-09T20:28:35.477 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcbf30df980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.10 2024-05-09T20:28:35.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcbf30df980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 : hit (requested=0x17, cached=0x17) 2024-05-09T20:28:35.478 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcbf30df980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 2024-05-09T20:28:35.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcbf30df980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:35.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcbf30df980 10 moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:35.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcbf30df980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:35.480 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcbf30df980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18, ofs0, ns] 2024-05-09T20:28:35.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcac77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431606 cookie 94464613232144 notifier 7074 bl.length()=781 2024-05-09T20:28:35.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcac77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:35.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcac77fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:35.481 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.487+0000 7fcac77fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:35.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.491+0000 7fcbf30df980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_sources= new_sources= 2024-05-09T20:28:35.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.491+0000 7fcbf30df980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential sources added=[] removed=[] 2024-05-09T20:28:35.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.491+0000 7fcbf30df980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_dests= new_dests= 2024-05-09T20:28:35.483 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.491+0000 7fcbf30df980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential dests added=[] removed=[] 2024-05-09T20:28:35.490 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.499+0000 7fcbf30df980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-05-09T20:28:35.493 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.503+0000 7fcaa77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:35.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.507+0000 7fcbf30df980 20 remove_watcher() i=0 2024-05-09T20:28:35.498 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.507+0000 7fcbf30df980 2 removed watcher, disabling cache 2024-05-09T20:28:35.500 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.507+0000 7fcbf30df980 20 remove_watcher() i=1 2024-05-09T20:28:35.504 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.511+0000 7fcbf30df980 20 remove_watcher() i=2 2024-05-09T20:28:35.505 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.515+0000 7fcbf30df980 20 remove_watcher() i=3 2024-05-09T20:28:35.507 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.515+0000 7fcbf30df980 20 remove_watcher() i=4 2024-05-09T20:28:35.509 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.519+0000 7fcbf30df980 20 remove_watcher() i=5 2024-05-09T20:28:35.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.519+0000 7fcbf30df980 20 remove_watcher() i=6 2024-05-09T20:28:35.512 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.519+0000 7fcbf30df980 20 remove_watcher() i=7 2024-05-09T20:28:35.530 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket myver 2024-05-09T20:28:35.561 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:35.561 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:35.607 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.615+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 realm 2024-05-09T20:28:35.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:35.609 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.610 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:35.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:35.612 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.619+0000 7f1d95745980 20 searching for the correct realm 2024-05-09T20:28:35.615 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.623+0000 7f1d93a5a640 0 -- 172.21.15.63:0/4094997773 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f1d7c0d0290 msgr2=0x7f1d7c0e5860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:35.624 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.631+0000 7f1d92a58640 0 -- 172.21.15.63:0/4094997773 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f1d8405d0f0 msgr2=0x7f1d8408b9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:35.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:35.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:35.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:35.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:35.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:35.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:35.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.636 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:35.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:35.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 zone default found 2024-05-09T20:28:35.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 4 Realm: () 2024-05-09T20:28:35.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:35.637 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:35.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:35.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 zonegroup default 2024-05-09T20:28:35.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.643+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.647+0000 7f1d95745980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.647+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.647+0000 7f1d95745980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.647+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.647+0000 7f1d95745980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:35.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.647+0000 7f1d95745980 20 started sync module instance, tier type = 2024-05-09T20:28:35.639 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.647+0000 7f1d95745980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:35.652 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.659+0000 7f1d95745980 20 add_watcher() i=0 2024-05-09T20:28:35.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.659+0000 7f1d95745980 20 add_watcher() i=1 2024-05-09T20:28:35.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.659+0000 7f1d95745980 20 add_watcher() i=2 2024-05-09T20:28:35.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.659+0000 7f1d95745980 20 add_watcher() i=3 2024-05-09T20:28:35.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.659+0000 7f1d95745980 20 add_watcher() i=4 2024-05-09T20:28:35.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.659+0000 7f1d95745980 20 add_watcher() i=5 2024-05-09T20:28:35.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.659+0000 7f1d95745980 20 add_watcher() i=6 2024-05-09T20:28:35.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1d95745980 20 add_watcher() i=7 2024-05-09T20:28:35.653 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1d95745980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:35.656 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1c49fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1d95745980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1d95745980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1d95745980 5 note: GC not initialized 2024-05-09T20:28:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1c48fe1640 20 reqs_thread_entry: start 2024-05-09T20:28:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1d95745980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1d95745980 20 Filter name: none 2024-05-09T20:28:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1bc27fc640 20 reqs_thread_entry: start 2024-05-09T20:28:35.657 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.663+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:35.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:35.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:35.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.658 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 rados->read ofs=0 len=0 2024-05-09T20:28:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:35.659 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.667+0000 7f1d95745980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:35.661 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.671+0000 7f1c49fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:35.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.675+0000 7f1d95745980 20 remove_watcher() i=0 2024-05-09T20:28:35.666 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.675+0000 7f1d95745980 2 removed watcher, disabling cache 2024-05-09T20:28:35.668 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.675+0000 7f1d95745980 20 remove_watcher() i=1 2024-05-09T20:28:35.670 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.679+0000 7f1d95745980 20 remove_watcher() i=2 2024-05-09T20:28:35.672 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.679+0000 7f1d95745980 20 remove_watcher() i=3 2024-05-09T20:28:35.674 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.683+0000 7f1d95745980 20 remove_watcher() i=4 2024-05-09T20:28:35.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.683+0000 7f1d95745980 20 remove_watcher() i=5 2024-05-09T20:28:35.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.683+0000 7f1d95745980 20 remove_watcher() i=6 2024-05-09T20:28:35.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:35.687+0000 7f1d95745980 20 remove_watcher() i=7 2024-05-09T20:28:35.690 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:35.690 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:28:35.690 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: myver id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 num_objects: 0 size_kb: 0 num_shards: 12 2024-05-09T20:28:35.691 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:35.774 INFO:tasks.workunit.client.0.smithi063.stderr: test: reshard removes olh entries with empty name 2024-05-09T20:28:36.182 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket a-bucket 2024-05-09T20:28:36.215 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:36.215 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:36.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 20 realm 2024-05-09T20:28:36.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:36.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.264 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.271+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.275+0000 7f58e7cde980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:36.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.275+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.275+0000 7f58e7cde980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:36.267 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.275+0000 7f58e7cde980 20 searching for the correct realm 2024-05-09T20:28:36.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.275+0000 7f58e4ff1640 0 -- 172.21.15.63:0/1040638853 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f58d8048ac0 msgr2=0x7f58d8036760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:36.479 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.487+0000 7f58e4ff1640 0 -- 172.21.15.63:0/1040638853 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f58d8067bd0 msgr2=0x7f58d803ed00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:36.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:36.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:36.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:36.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:36.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:36.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:36.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:36.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.711+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:36.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 zone default found 2024-05-09T20:28:36.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 4 Realm: () 2024-05-09T20:28:36.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:36.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:36.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:36.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 zonegroup default 2024-05-09T20:28:36.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.707 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 started sync module instance, tier type = 2024-05-09T20:28:36.708 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.715+0000 7f58e7cde980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:36.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 20 add_watcher() i=0 2024-05-09T20:28:36.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 20 add_watcher() i=1 2024-05-09T20:28:36.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 20 add_watcher() i=2 2024-05-09T20:28:36.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 20 add_watcher() i=3 2024-05-09T20:28:36.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 20 add_watcher() i=4 2024-05-09T20:28:36.733 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 20 add_watcher() i=5 2024-05-09T20:28:36.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 20 add_watcher() i=6 2024-05-09T20:28:36.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 20 add_watcher() i=7 2024-05-09T20:28:36.736 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.743+0000 7f58e7cde980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:36.740 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.747+0000 7f579bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:36.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f58e7cde980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:36.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f58e7cde980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:36.741 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f58e7cde980 5 note: GC not initialized 2024-05-09T20:28:36.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f579afe5640 20 reqs_thread_entry: start 2024-05-09T20:28:36.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f58e7cde980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:36.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f58e7cde980 20 Filter name: none 2024-05-09T20:28:36.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f5798fe1640 20 reqs_thread_entry: start 2024-05-09T20:28:36.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f58e7cde980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:28:36.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.751+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.755+0000 7f58e7cde980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:28:36.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.755+0000 7f58e7cde980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:36.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.755+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.755+0000 7f58e7cde980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:28:36.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.755+0000 7f58e7cde980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.755+0000 7f58e7cde980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:28:36.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.755+0000 7f58e7cde980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:36.750 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.759+0000 7f579bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:36.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.763+0000 7f58e7cde980 20 remove_watcher() i=0 2024-05-09T20:28:36.755 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.763+0000 7f58e7cde980 2 removed watcher, disabling cache 2024-05-09T20:28:36.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.767+0000 7f58e7cde980 20 remove_watcher() i=1 2024-05-09T20:28:36.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.767+0000 7f58e7cde980 20 remove_watcher() i=2 2024-05-09T20:28:36.762 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.771+0000 7f58e7cde980 20 remove_watcher() i=3 2024-05-09T20:28:36.764 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.771+0000 7f58e7cde980 20 remove_watcher() i=4 2024-05-09T20:28:36.766 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.775+0000 7f58e7cde980 20 remove_watcher() i=5 2024-05-09T20:28:36.767 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.775+0000 7f58e7cde980 20 remove_watcher() i=6 2024-05-09T20:28:36.768 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.775+0000 7f58e7cde980 20 remove_watcher() i=7 2024-05-09T20:28:36.786 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:36.786 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:28:36.786 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: a-bucket id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 num_objects: 0 size_kb: 0 num_shards: 13 2024-05-09T20:28:36.786 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:36.787 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket layout --bucket a-bucket 2024-05-09T20:28:36.813 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:36.814 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:36.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.867+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.867+0000 7fb384d3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.867+0000 7fb384d3e980 20 realm 2024-05-09T20:28:36.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.867+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.861 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:36.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:36.862 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:36.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:36.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:36.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.871+0000 7fb384d3e980 20 searching for the correct realm 2024-05-09T20:28:36.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:36.875+0000 7fb382852640 0 -- 172.21.15.63:0/3700833326 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fb37c0cf410 msgr2=0x7fb37c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:37.082 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.091+0000 7fb382852640 0 -- 172.21.15.63:0/3700833326 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fb37c0cf410 msgr2=0x7fb37c0cf870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:37.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.099+0000 7fb384d3e980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:37.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.099+0000 7fb384d3e980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:37.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.099+0000 7fb384d3e980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:37.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.099+0000 7fb384d3e980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:37.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.099+0000 7fb384d3e980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:37.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.099+0000 7fb384d3e980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:37.093 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.099+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:37.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.095 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:37.095 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 zone default found 2024-05-09T20:28:37.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 4 Realm: () 2024-05-09T20:28:37.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:37.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:37.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:37.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 zonegroup default 2024-05-09T20:28:37.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.096 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.103+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.107+0000 7fb384d3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.107+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.107+0000 7fb384d3e980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.107+0000 7fb384d3e980 20 started sync module instance, tier type = 2024-05-09T20:28:37.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.107+0000 7fb384d3e980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:37.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.127+0000 7fb384d3e980 20 add_watcher() i=0 2024-05-09T20:28:37.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.127+0000 7fb384d3e980 20 add_watcher() i=1 2024-05-09T20:28:37.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.127+0000 7fb384d3e980 20 add_watcher() i=2 2024-05-09T20:28:37.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.127+0000 7fb384d3e980 20 add_watcher() i=3 2024-05-09T20:28:37.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.127+0000 7fb384d3e980 20 add_watcher() i=4 2024-05-09T20:28:37.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.127+0000 7fb384d3e980 20 add_watcher() i=5 2024-05-09T20:28:37.121 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.127+0000 7fb384d3e980 20 add_watcher() i=6 2024-05-09T20:28:37.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.131+0000 7fb384d3e980 20 add_watcher() i=7 2024-05-09T20:28:37.123 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.131+0000 7fb384d3e980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:37.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.135+0000 7fb2397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:37.128 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.135+0000 7fb384d3e980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:37.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.135+0000 7fb384d3e980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:37.129 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.135+0000 7fb384d3e980 5 note: GC not initialized 2024-05-09T20:28:37.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.135+0000 7fb1b3fff640 20 reqs_thread_entry: start 2024-05-09T20:28:37.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb384d3e980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:37.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb384d3e980 20 Filter name: none 2024-05-09T20:28:37.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb1b1ffb640 20 reqs_thread_entry: start 2024-05-09T20:28:37.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb384d3e980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:28:37.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb384d3e980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb384d3e980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:28:37.131 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb384d3e980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:37.133 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.139+0000 7fb2397e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:37.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.147+0000 7fb384d3e980 20 remove_watcher() i=0 2024-05-09T20:28:37.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.147+0000 7fb384d3e980 2 removed watcher, disabling cache 2024-05-09T20:28:37.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.151+0000 7fb384d3e980 20 remove_watcher() i=1 2024-05-09T20:28:37.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.151+0000 7fb384d3e980 20 remove_watcher() i=2 2024-05-09T20:28:37.146 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.155+0000 7fb384d3e980 20 remove_watcher() i=3 2024-05-09T20:28:37.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.155+0000 7fb384d3e980 20 remove_watcher() i=4 2024-05-09T20:28:37.149 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.159+0000 7fb384d3e980 20 remove_watcher() i=5 2024-05-09T20:28:37.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.159+0000 7fb384d3e980 20 remove_watcher() i=6 2024-05-09T20:28:37.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.159+0000 7fb384d3e980 20 remove_watcher() i=7 2024-05-09T20:28:37.169 INFO:tasks.workunit.client.0.smithi063.stderr:rados -p default.rgw.buckets.index setomapval .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.0 --omap-key-file /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_key < /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/olh_noname_val 2024-05-09T20:28:37.201 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:37.202 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:37.220 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket a-bucket 2024-05-09T20:28:37.246 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:37.246 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:37.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.295+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 realm 2024-05-09T20:28:37.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:37.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:37.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.299+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.303+0000 7f2bd7fb9980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:37.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.303+0000 7f2bd7fb9980 20 searching for the correct realm 2024-05-09T20:28:37.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.303+0000 7f2bd62ce640 0 -- 172.21.15.63:0/761001253 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f2bcc03f020 msgr2=0x7f2bcc066720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:37.510 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.515+0000 7f2bd52cc640 0 -- 172.21.15.63:0/761001253 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f2bc8008050 msgr2=0x7f2bc8055920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:37.525 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.531+0000 7f2bd7fb9980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:37.525 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.531+0000 7f2bd7fb9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:37.525 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.531+0000 7f2bd7fb9980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:37.525 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.531+0000 7f2bd7fb9980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:37.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.531+0000 7f2bd7fb9980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:37.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.531+0000 7f2bd7fb9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:37.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.531+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.526 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:37.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:37.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 zone default found 2024-05-09T20:28:37.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 4 Realm: () 2024-05-09T20:28:37.527 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:37.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:37.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:37.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 zonegroup default 2024-05-09T20:28:37.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.535+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.539+0000 7f2bd7fb9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.529 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.539+0000 7f2bd7fb9980 20 started sync module instance, tier type = 2024-05-09T20:28:37.530 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.539+0000 7f2bd7fb9980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:37.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 20 add_watcher() i=0 2024-05-09T20:28:37.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 20 add_watcher() i=1 2024-05-09T20:28:37.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 20 add_watcher() i=2 2024-05-09T20:28:37.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 20 add_watcher() i=3 2024-05-09T20:28:37.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 20 add_watcher() i=4 2024-05-09T20:28:37.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 20 add_watcher() i=5 2024-05-09T20:28:37.554 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 20 add_watcher() i=6 2024-05-09T20:28:37.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 20 add_watcher() i=7 2024-05-09T20:28:37.556 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.563+0000 7f2bd7fb9980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:37.560 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.567+0000 7f2a8bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.567+0000 7f2bd7fb9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.567+0000 7f2bd7fb9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:37.561 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 5 note: GC not initialized 2024-05-09T20:28:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2a8afe5640 20 reqs_thread_entry: start 2024-05-09T20:28:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 Filter name: none 2024-05-09T20:28:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2a88fe1640 20 reqs_thread_entry: start 2024-05-09T20:28:37.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:28:37.563 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:28:37.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:37.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-05-09T20:28:37.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 INFO: main: starting shard=0 2024-05-09T20:28:37.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.0 2024-05-09T20:28:37.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.0 2024-05-09T20:28:37.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.571+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-09T20:28:37.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 INFO: main: starting shard=1 2024-05-09T20:28:37.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.1 2024-05-09T20:28:37.565 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.1 2024-05-09T20:28:37.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 INFO: main: starting shard=2 2024-05-09T20:28:37.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.2 2024-05-09T20:28:37.566 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.2 2024-05-09T20:28:37.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 INFO: main: starting shard=3 2024-05-09T20:28:37.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.3 2024-05-09T20:28:37.567 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.3 2024-05-09T20:28:37.568 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.568 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 INFO: main: starting shard=4 2024-05-09T20:28:37.568 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.4 2024-05-09T20:28:37.568 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.575+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.4 2024-05-09T20:28:37.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 INFO: main: starting shard=5 2024-05-09T20:28:37.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.5 2024-05-09T20:28:37.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.5 2024-05-09T20:28:37.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 INFO: main: starting shard=6 2024-05-09T20:28:37.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.6 2024-05-09T20:28:37.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.6 2024-05-09T20:28:37.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 INFO: main: starting shard=7 2024-05-09T20:28:37.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.7 2024-05-09T20:28:37.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.579+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.7 2024-05-09T20:28:37.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 INFO: main: starting shard=8 2024-05-09T20:28:37.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.8 2024-05-09T20:28:37.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.8 2024-05-09T20:28:37.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 INFO: main: starting shard=9 2024-05-09T20:28:37.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.9 2024-05-09T20:28:37.574 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.9 2024-05-09T20:28:37.575 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 INFO: main: starting shard=10 2024-05-09T20:28:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.10 2024-05-09T20:28:37.576 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.10 2024-05-09T20:28:37.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 INFO: main: starting shard=11 2024-05-09T20:28:37.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.11 2024-05-09T20:28:37.577 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.583+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.11 2024-05-09T20:28:37.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.587+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.587+0000 7f2bd7fb9980 20 INFO: main: starting shard=12 2024-05-09T20:28:37.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.587+0000 7f2bd7fb9980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.12 2024-05-09T20:28:37.578 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.587+0000 7f2bd7fb9980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.12 2024-05-09T20:28:37.579 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.587+0000 7f2bd7fb9980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:37.579 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.587+0000 7f2bd7fb9980 20 INFO: main: done 2024-05-09T20:28:37.580 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.587+0000 7f2a8bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:37.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.595+0000 7f2bd7fb9980 20 remove_watcher() i=0 2024-05-09T20:28:37.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.595+0000 7f2bd7fb9980 2 removed watcher, disabling cache 2024-05-09T20:28:37.587 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.595+0000 7f2bd7fb9980 20 remove_watcher() i=1 2024-05-09T20:28:37.590 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.599+0000 7f2bd7fb9980 20 remove_watcher() i=2 2024-05-09T20:28:37.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.599+0000 7f2bd7fb9980 20 remove_watcher() i=3 2024-05-09T20:28:37.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.603+0000 7f2bd7fb9980 20 remove_watcher() i=4 2024-05-09T20:28:37.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.603+0000 7f2bd7fb9980 20 remove_watcher() i=5 2024-05-09T20:28:37.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.607+0000 7f2bd7fb9980 20 remove_watcher() i=6 2024-05-09T20:28:37.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.607+0000 7f2bd7fb9980 20 remove_watcher() i=7 2024-05-09T20:28:37.618 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket a-bucket --num-shards 1 --yes-i-really-mean-it 2024-05-09T20:28:37.654 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:37.654 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:37.698 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 20 realm 2024-05-09T20:28:37.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.700 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:37.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.701 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.707+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.711+0000 7fa8e75c8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:37.702 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.711+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.711+0000 7fa8e75c8980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:37.703 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.711+0000 7fa8e75c8980 20 searching for the correct realm 2024-05-09T20:28:37.706 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.715+0000 7fa8e58dd640 0 -- 172.21.15.63:0/3502359749 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fa8dc0f5820 msgr2=0x7fa8dc0bf700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:37.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.927+0000 7fa8e48db640 0 -- 172.21.15.63:0/3502359749 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fa8d801e550 msgr2=0x7fa8d802d440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:37.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.943+0000 7fa8e75c8980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:37.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.943+0000 7fa8e75c8980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:37.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.943+0000 7fa8e75c8980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:37.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.943+0000 7fa8e75c8980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:37.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.943+0000 7fa8e75c8980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:37.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.943+0000 7fa8e75c8980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:37.937 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.943+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:37.938 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 zone default found 2024-05-09T20:28:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 4 Realm: () 2024-05-09T20:28:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 zonegroup default 2024-05-09T20:28:37.939 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.940 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.947+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.941 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.951+0000 7fa8e75c8980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:37.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.951+0000 7fa8e75c8980 20 started sync module instance, tier type = 2024-05-09T20:28:37.942 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.951+0000 7fa8e75c8980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:37.966 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 20 add_watcher() i=0 2024-05-09T20:28:37.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 20 add_watcher() i=1 2024-05-09T20:28:37.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 20 add_watcher() i=2 2024-05-09T20:28:37.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 20 add_watcher() i=3 2024-05-09T20:28:37.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 20 add_watcher() i=4 2024-05-09T20:28:37.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 20 add_watcher() i=5 2024-05-09T20:28:37.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 20 add_watcher() i=6 2024-05-09T20:28:37.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 20 add_watcher() i=7 2024-05-09T20:28:37.969 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.975+0000 7fa8e75c8980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:37.973 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.979+0000 7fa79bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:37.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:37.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:37.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 5 note: GC not initialized 2024-05-09T20:28:37.974 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa79afe5640 20 reqs_thread_entry: start 2024-05-09T20:28:37.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:37.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 20 Filter name: none 2024-05-09T20:28:37.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa798fe1640 20 reqs_thread_entry: start 2024-05-09T20:28:37.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 cache get: name=default.rgw.meta+root+a-bucket : miss 2024-05-09T20:28:37.975 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:28:37.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 cache put: name=default.rgw.meta+root+a-bucket info.flags=0x11 2024-05-09T20:28:37.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 adding default.rgw.meta+root+a-bucket to cache LRU end 2024-05-09T20:28:37.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 : miss 2024-05-09T20:28:37.976 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 20 rados->read ofs=0 len=0 2024-05-09T20:28:37.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:28:37.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:28:37.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 adding default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:28:37.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:37.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:37.977 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.983+0000 7fa8e75c8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 2024-05-09T20:28:37.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa8e75c8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:28:37.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa8e75c8980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:28:37.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa8e75c8980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:37.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa8e75c8980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17, ofs0, ns] 2024-05-09T20:28:37.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa7b7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431607 cookie 94188370059568 notifier 6755 bl.length()=819 2024-05-09T20:28:37.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa7b7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:28:37.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa7b7fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:28:37.986 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa7b7fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:37.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa8e75c8980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_sources= new_sources= 2024-05-09T20:28:37.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa8e75c8980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential sources added=[] removed=[] 2024-05-09T20:28:37.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa8e75c8980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_dests= new_dests= 2024-05-09T20:28:37.988 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:37.995+0000 7fa8e75c8980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential dests added=[] removed=[] 2024-05-09T20:28:37.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.3.0 2024-05-09T20:28:37.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.3.0 2024-05-09T20:28:37.994 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.0 2024-05-09T20:28:37.998 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.0 2024-05-09T20:28:37.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 10 Dropping entry with empty name, idx=�1001_04/57/0457f727ec113e418d5b16d206b200ed068c0533554883ce811df7c932a3df68/2018_12_11/2889999/3386469/metadata.gz 2024-05-09T20:28:37.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.1 2024-05-09T20:28:37.999 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.1 2024-05-09T20:28:38.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.2 2024-05-09T20:28:38.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.2 2024-05-09T20:28:38.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.3 2024-05-09T20:28:38.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.003+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.3 2024-05-09T20:28:38.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.007+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.4 2024-05-09T20:28:38.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.007+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.4 2024-05-09T20:28:38.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.007+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.5 2024-05-09T20:28:38.000 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.007+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.5 2024-05-09T20:28:38.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.007+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.6 2024-05-09T20:28:38.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.007+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.6 2024-05-09T20:28:38.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.7 2024-05-09T20:28:38.001 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.7 2024-05-09T20:28:38.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.8 2024-05-09T20:28:38.002 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.8 2024-05-09T20:28:38.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.9 2024-05-09T20:28:38.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.9 2024-05-09T20:28:38.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.10 2024-05-09T20:28:38.003 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.10 2024-05-09T20:28:38.004 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.11 2024-05-09T20:28:38.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.011+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.11 2024-05-09T20:28:38.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.015+0000 7fa8e75c8980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.12 2024-05-09T20:28:38.005 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.015+0000 7fa8e75c8980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.2.12 2024-05-09T20:28:38.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.015+0000 7fa8e75c8980 10 cache get: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 : hit (requested=0x17, cached=0x17) 2024-05-09T20:28:38.007 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.015+0000 7fa8e75c8980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 2024-05-09T20:28:38.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa8e75c8980 10 cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:28:38.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa8e75c8980 10 moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:28:38.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa8e75c8980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:38.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa8e75c8980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17, ofs0, ns] 2024-05-09T20:28:38.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa7c8ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431608 cookie 94188370059568 notifier 6755 bl.length()=787 2024-05-09T20:28:38.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa7c8ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 info.flags=0x17 2024-05-09T20:28:38.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa7c8ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.a-bucket:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17 to cache LRU end 2024-05-09T20:28:38.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa7c8ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:38.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa8e75c8980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_sources= new_sources= 2024-05-09T20:28:38.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa8e75c8980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential sources added=[] removed=[] 2024-05-09T20:28:38.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa8e75c8980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): orig_dests= new_dests= 2024-05-09T20:28:38.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.019+0000 7fa8e75c8980 20 handle_bi_update(): bucket=:a-bucket[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17]): potential dests added=[] removed=[] 2024-05-09T20:28:38.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.027+0000 7fa8e75c8980 1 execute INFO: reshard of bucket "a-bucket" completed successfully 2024-05-09T20:28:38.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.031+0000 7fa79bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:38.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.035+0000 7fa8e75c8980 20 remove_watcher() i=0 2024-05-09T20:28:38.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.035+0000 7fa8e75c8980 2 removed watcher, disabling cache 2024-05-09T20:28:38.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.039+0000 7fa8e75c8980 20 remove_watcher() i=1 2024-05-09T20:28:38.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.043+0000 7fa8e75c8980 20 remove_watcher() i=2 2024-05-09T20:28:38.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.043+0000 7fa8e75c8980 20 remove_watcher() i=3 2024-05-09T20:28:38.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.047+0000 7fa8e75c8980 20 remove_watcher() i=4 2024-05-09T20:28:38.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.047+0000 7fa8e75c8980 20 remove_watcher() i=5 2024-05-09T20:28:38.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.051+0000 7fa8e75c8980 20 remove_watcher() i=6 2024-05-09T20:28:38.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.051+0000 7fa8e75c8980 20 remove_watcher() i=7 2024-05-09T20:28:38.060 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket a-bucket 2024-05-09T20:28:38.090 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:38.091 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:38.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 20 realm 2024-05-09T20:28:38.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.136 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:38.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.143+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.147+0000 7f1aa3355980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:38.138 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.147+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.147+0000 7f1aa3355980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:38.140 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.147+0000 7f1aa3355980 20 searching for the correct realm 2024-05-09T20:28:38.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.151+0000 7f1aa0e69640 0 -- 172.21.15.63:0/151604342 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f1a9800e040 msgr2=0x7f1a9807fa50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:38.360 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.367+0000 7f1aa0e69640 0 -- 172.21.15.63:0/151604342 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f1a9800e040 msgr2=0x7f1a9807fa50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:38.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:38.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:38.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:38.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:38.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:38.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:38.372 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.379+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:38.373 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:38.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 zone default found 2024-05-09T20:28:38.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 4 Realm: () 2024-05-09T20:28:38.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:38.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:38.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:38.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 zonegroup default 2024-05-09T20:28:38.374 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.375 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 started sync module instance, tier type = 2024-05-09T20:28:38.376 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.383+0000 7f1aa3355980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:38.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.407+0000 7f1aa3355980 20 add_watcher() i=0 2024-05-09T20:28:38.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.407+0000 7f1aa3355980 20 add_watcher() i=1 2024-05-09T20:28:38.398 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.407+0000 7f1aa3355980 20 add_watcher() i=2 2024-05-09T20:28:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.407+0000 7f1aa3355980 20 add_watcher() i=3 2024-05-09T20:28:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.407+0000 7f1aa3355980 20 add_watcher() i=4 2024-05-09T20:28:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.407+0000 7f1aa3355980 20 add_watcher() i=5 2024-05-09T20:28:38.399 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.407+0000 7f1aa3355980 20 add_watcher() i=6 2024-05-09T20:28:38.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.411+0000 7f1aa3355980 20 add_watcher() i=7 2024-05-09T20:28:38.401 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.411+0000 7f1aa3355980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:38.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f19577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:38.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f1aa3355980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:38.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f1aa3355980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:38.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f1aa3355980 5 note: GC not initialized 2024-05-09T20:28:38.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f19567e4640 20 reqs_thread_entry: start 2024-05-09T20:28:38.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f1aa3355980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:38.407 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f1aa3355980 20 Filter name: none 2024-05-09T20:28:38.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f19547e0640 20 reqs_thread_entry: start 2024-05-09T20:28:38.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f1aa3355980 20 rados_obj.operate() r=0 bl.length=176 2024-05-09T20:28:38.408 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.415+0000 7f1aa3355980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f1aa3355980 20 rados_obj.operate() r=0 bl.length=390 2024-05-09T20:28:38.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f1aa3355980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:38.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f1aa3355980 20 INFO: main: max_entries=1000, index=gen=3, max_shards=1 2024-05-09T20:28:38.409 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f1aa3355980 20 INFO: main: starting shard=0 2024-05-09T20:28:38.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f1aa3355980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.3.0 2024-05-09T20:28:38.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f1aa3355980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17.3.0 2024-05-09T20:28:38.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f1aa3355980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:38.410 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f1aa3355980 20 INFO: main: done 2024-05-09T20:28:38.411 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.419+0000 7f19577e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:38.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.423+0000 7f1aa3355980 20 remove_watcher() i=0 2024-05-09T20:28:38.417 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.423+0000 7f1aa3355980 2 removed watcher, disabling cache 2024-05-09T20:28:38.419 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.427+0000 7f1aa3355980 20 remove_watcher() i=1 2024-05-09T20:28:38.421 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.431+0000 7f1aa3355980 20 remove_watcher() i=2 2024-05-09T20:28:38.424 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.431+0000 7f1aa3355980 20 remove_watcher() i=3 2024-05-09T20:28:38.426 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.435+0000 7f1aa3355980 20 remove_watcher() i=4 2024-05-09T20:28:38.428 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.435+0000 7f1aa3355980 20 remove_watcher() i=5 2024-05-09T20:28:38.429 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.439+0000 7f1aa3355980 20 remove_watcher() i=6 2024-05-09T20:28:38.431 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.439+0000 7f1aa3355980 20 remove_watcher() i=7 2024-05-09T20:28:38.447 INFO:tasks.workunit.client.0.smithi063.stderr: test: PUT succeeds during reshard 2024-05-09T20:28:38.448 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket myver 2024-05-09T20:28:38.475 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:38.475 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:38.519 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.527+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.521 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 realm 2024-05-09T20:28:38.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:38.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.522 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.523 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:38.523 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:38.524 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.531+0000 7ff796027980 20 searching for the correct realm 2024-05-09T20:28:38.528 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.535+0000 7ff793b3b640 0 -- 172.21.15.63:0/1216278502 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7ff784008050 msgr2=0x7ff784008430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:38.743 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.751+0000 7ff793b3b640 0 -- 172.21.15.63:0/1216278502 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7ff784008050 msgr2=0x7ff784008430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.756 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:38.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.763+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:38.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 zone default found 2024-05-09T20:28:38.757 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 4 Realm: () 2024-05-09T20:28:38.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:38.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:38.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:38.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 zonegroup default 2024-05-09T20:28:38.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.758 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.759 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 started sync module instance, tier type = 2024-05-09T20:28:38.760 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.767+0000 7ff796027980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:38.784 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.791+0000 7ff796027980 20 add_watcher() i=0 2024-05-09T20:28:38.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.791+0000 7ff796027980 20 add_watcher() i=1 2024-05-09T20:28:38.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.791+0000 7ff796027980 20 add_watcher() i=2 2024-05-09T20:28:38.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.791+0000 7ff796027980 20 add_watcher() i=3 2024-05-09T20:28:38.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.791+0000 7ff796027980 20 add_watcher() i=4 2024-05-09T20:28:38.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.791+0000 7ff796027980 20 add_watcher() i=5 2024-05-09T20:28:38.785 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.791+0000 7ff796027980 20 add_watcher() i=6 2024-05-09T20:28:38.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.795+0000 7ff796027980 20 add_watcher() i=7 2024-05-09T20:28:38.787 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.795+0000 7ff796027980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:38.791 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.799+0000 7ff649fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:38.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.799+0000 7ff796027980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:38.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.799+0000 7ff796027980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:38.792 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.799+0000 7ff796027980 5 note: GC not initialized 2024-05-09T20:28:38.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.799+0000 7ff648fe1640 20 reqs_thread_entry: start 2024-05-09T20:28:38.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.799+0000 7ff796027980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:38.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.799+0000 7ff796027980 20 Filter name: none 2024-05-09T20:28:38.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff5c27fc640 20 reqs_thread_entry: start 2024-05-09T20:28:38.793 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:38.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:38.794 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:38.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:38.795 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:38.796 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.803+0000 7ff796027980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:38.799 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.807+0000 7ff649fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:38.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.811+0000 7ff796027980 20 remove_watcher() i=0 2024-05-09T20:28:38.804 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.811+0000 7ff796027980 2 removed watcher, disabling cache 2024-05-09T20:28:38.807 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.815+0000 7ff796027980 20 remove_watcher() i=1 2024-05-09T20:28:38.809 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.819+0000 7ff796027980 20 remove_watcher() i=2 2024-05-09T20:28:38.812 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.819+0000 7ff796027980 20 remove_watcher() i=3 2024-05-09T20:28:38.814 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.823+0000 7ff796027980 20 remove_watcher() i=4 2024-05-09T20:28:38.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.823+0000 7ff796027980 20 remove_watcher() i=5 2024-05-09T20:28:38.818 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.827+0000 7ff796027980 20 remove_watcher() i=6 2024-05-09T20:28:38.819 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.827+0000 7ff796027980 20 remove_watcher() i=7 2024-05-09T20:28:38.836 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:38.836 INFO:tasks.workunit.client.0.smithi063.stderr:BUCKET_STATS: 2024-05-09T20:28:38.836 INFO:tasks.workunit.client.0.smithi063.stderr:bucket: myver id: 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 num_objects: 0 size_kb: 0 num_shards: 12 2024-05-09T20:28:38.836 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:38.836 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin --inject-delay-at=do_reshard --inject-delay-ms=5000 bucket reshard --bucket myver --num-shards 13 2024-05-09T20:28:38.869 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:38.869 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:38.913 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.919+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 realm 2024-05-09T20:28:38.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:38.914 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:38.915 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:38.916 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.923+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:38.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.927+0000 7f798e899980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:38.917 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.927+0000 7f798e899980 20 searching for the correct realm 2024-05-09T20:28:38.921 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:38.927+0000 7f798c3ad640 0 -- 172.21.15.63:0/512096778 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f79840f8780 msgr2=0x7f79840fa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:39.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.147+0000 7f798c3ad640 0 -- 172.21.15.63:0/512096778 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f79840f8780 msgr2=0x7f79840fa810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:39.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:39.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:39.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:39.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:39.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:39.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:39.151 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:39.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:39.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:39.152 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:39.153 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.159+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:39.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:39.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 zone default found 2024-05-09T20:28:39.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 4 Realm: () 2024-05-09T20:28:39.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:39.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:39.154 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:39.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 zonegroup default 2024-05-09T20:28:39.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:39.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:39.155 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:39.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:39.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:39.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:39.156 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 started sync module instance, tier type = 2024-05-09T20:28:39.157 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.163+0000 7f798e899980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:39.181 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 20 add_watcher() i=0 2024-05-09T20:28:39.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 20 add_watcher() i=1 2024-05-09T20:28:39.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 20 add_watcher() i=2 2024-05-09T20:28:39.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 20 add_watcher() i=3 2024-05-09T20:28:39.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 20 add_watcher() i=4 2024-05-09T20:28:39.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 20 add_watcher() i=5 2024-05-09T20:28:39.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 20 add_watcher() i=6 2024-05-09T20:28:39.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 20 add_watcher() i=7 2024-05-09T20:28:39.184 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.191+0000 7f798e899980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:39.188 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.195+0000 7f78437de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:39.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.195+0000 7f798e899980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:39.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.195+0000 7f798e899980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:39.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 5 note: GC not initialized 2024-05-09T20:28:39.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f78427dc640 20 reqs_thread_entry: start 2024-05-09T20:28:39.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:39.190 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 20 Filter name: none 2024-05-09T20:28:39.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f783b7fe640 20 reqs_thread_entry: start 2024-05-09T20:28:39.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-05-09T20:28:39.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:39.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:39.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-05-09T20:28:39.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-05-09T20:28:39.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 : miss 2024-05-09T20:28:39.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 20 rados->read ofs=0 len=0 2024-05-09T20:28:39.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:39.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:39.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 adding default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:39.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:39.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:39.193 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.199+0000 7f798e899980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 2024-05-09T20:28:39.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f798e899980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:39.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f798e899980 10 moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:39.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f798e899980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:39.202 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f798e899980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18, ofs0, ns] 2024-05-09T20:28:39.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f78637fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431609 cookie 94705797581952 notifier 7227 bl.length()=813 2024-05-09T20:28:39.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f78637fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:39.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f78637fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:39.203 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f78637fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:39.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.211+0000 7f798e899980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_sources= new_sources= 2024-05-09T20:28:39.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.215+0000 7f798e899980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential sources added=[] removed=[] 2024-05-09T20:28:39.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.215+0000 7f798e899980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_dests= new_dests= 2024-05-09T20:28:39.205 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.215+0000 7f798e899980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential dests added=[] removed=[] 2024-05-09T20:28:39.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:39.219+0000 7f798e899980 -1 Injecting delay=5.000000000s at location=do_reshard 2024-05-09T20:28:44.210 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:44.211 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:44.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:44.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:44.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:44.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:44.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:44.212 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:44.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:44.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:44.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:44.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:44.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:44.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:44.213 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:44.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:44.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:44.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:44.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.0 2024-05-09T20:28:44.214 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.0 2024-05-09T20:28:44.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.1 2024-05-09T20:28:44.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.1 2024-05-09T20:28:44.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.2 2024-05-09T20:28:44.216 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.219+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.2 2024-05-09T20:28:44.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.223+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.3 2024-05-09T20:28:44.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.223+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.3 2024-05-09T20:28:44.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.223+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.4 2024-05-09T20:28:44.217 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.223+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.4 2024-05-09T20:28:44.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.227+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.5 2024-05-09T20:28:44.218 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.227+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.5 2024-05-09T20:28:44.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.227+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.6 2024-05-09T20:28:44.219 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.227+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.6 2024-05-09T20:28:44.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.227+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.7 2024-05-09T20:28:44.220 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.227+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.7 2024-05-09T20:28:44.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.231+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.8 2024-05-09T20:28:44.221 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.231+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.8 2024-05-09T20:28:44.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.231+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.9 2024-05-09T20:28:44.223 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.231+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.9 2024-05-09T20:28:44.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.231+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.10 2024-05-09T20:28:44.224 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.231+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.10 2024-05-09T20:28:44.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.235+0000 7f798e899980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.11 2024-05-09T20:28:44.226 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.235+0000 7f798e899980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.1.11 2024-05-09T20:28:44.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.235+0000 7f798e899980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 : hit (requested=0x17, cached=0x17) 2024-05-09T20:28:44.228 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.235+0000 7f798e899980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 2024-05-09T20:28:44.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.239+0000 7f798e899980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:44.231 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.239+0000 7f798e899980 10 moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:44.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.239+0000 7f798e899980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:44.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.239+0000 7f798e899980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18, ofs0, ns] 2024-05-09T20:28:44.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.239+0000 7f78637fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431610 cookie 94705797581952 notifier 7227 bl.length()=781 2024-05-09T20:28:44.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.239+0000 7f78637fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:44.232 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.239+0000 7f78637fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:44.233 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.239+0000 7f78637fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:44.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.243+0000 7f798e899980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_sources= new_sources= 2024-05-09T20:28:44.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.243+0000 7f798e899980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential sources added=[] removed=[] 2024-05-09T20:28:44.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.243+0000 7f798e899980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_dests= new_dests= 2024-05-09T20:28:44.236 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.243+0000 7f798e899980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential dests added=[] removed=[] 2024-05-09T20:28:44.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.251+0000 7f798e899980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-05-09T20:28:44.250 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.259+0000 7f78437de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:44.255 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.263+0000 7f798e899980 20 remove_watcher() i=0 2024-05-09T20:28:44.256 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.263+0000 7f798e899980 2 removed watcher, disabling cache 2024-05-09T20:28:44.258 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.267+0000 7f798e899980 20 remove_watcher() i=1 2024-05-09T20:28:44.261 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.271+0000 7f798e899980 20 remove_watcher() i=2 2024-05-09T20:28:44.263 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.271+0000 7f798e899980 20 remove_watcher() i=3 2024-05-09T20:28:44.266 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.275+0000 7f798e899980 20 remove_watcher() i=4 2024-05-09T20:28:44.268 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.275+0000 7f798e899980 20 remove_watcher() i=5 2024-05-09T20:28:44.271 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.279+0000 7f798e899980 20 remove_watcher() i=6 2024-05-09T20:28:44.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:44.279+0000 7f798e899980 20 remove_watcher() i=7 2024-05-09T20:28:45.067 INFO:tasks.workunit.client.0.smithi063.stderr:put object successful 2024-05-09T20:28:45.067 INFO:tasks.workunit.client.0.smithi063.stderr:TEST: check that bucket stats are correct after reshard with unlinked entries 2024-05-09T20:28:45.067 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-05-09T20:28:46.023 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config set client rgw_debug_inject_set_olh_err 2 2024-05-09T20:28:46.460 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config set client rgw_debug_inject_olh_cancel_modification_err true 2024-05-09T20:28:48.034 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:28:48.035 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket myver --object x 2024-05-09T20:28:48.071 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:48.071 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:48.115 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.123+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.123+0000 7fe246765980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.123+0000 7fe246765980 20 realm 2024-05-09T20:28:48.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.123+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.117 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.123+0000 7fe246765980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.123+0000 7fe246765980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:48.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.123+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.127+0000 7fe246765980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.118 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.127+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.127+0000 7fe246765980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:48.119 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.127+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.127+0000 7fe246765980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:48.120 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.127+0000 7fe246765980 20 searching for the correct realm 2024-05-09T20:28:48.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.131+0000 7fe244a7a640 0 -- 172.21.15.63:0/2264363333 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fe238037030 msgr2=0x7fe238040910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:48.130 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.139+0000 7fe243a78640 0 -- 172.21.15.63:0/2264363333 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x55eccc94fc20 msgr2=0x7fe2340c2ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.147+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 zone default found 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 4 Realm: () 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 zonegroup default 2024-05-09T20:28:48.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 started sync module instance, tier type = 2024-05-09T20:28:48.144 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.151+0000 7fe246765980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:48.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 20 add_watcher() i=0 2024-05-09T20:28:48.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 20 add_watcher() i=1 2024-05-09T20:28:48.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 20 add_watcher() i=2 2024-05-09T20:28:48.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 20 add_watcher() i=3 2024-05-09T20:28:48.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 20 add_watcher() i=4 2024-05-09T20:28:48.160 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 20 add_watcher() i=5 2024-05-09T20:28:48.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 20 add_watcher() i=6 2024-05-09T20:28:48.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 20 add_watcher() i=7 2024-05-09T20:28:48.161 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.167+0000 7fe246765980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe0fa7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe246765980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe246765980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe246765980 5 note: GC not initialized 2024-05-09T20:28:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe0f97e2640 20 reqs_thread_entry: start 2024-05-09T20:28:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe246765980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe246765980 20 Filter name: none 2024-05-09T20:28:48.165 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe06affd640 20 reqs_thread_entry: start 2024-05-09T20:28:48.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe246765980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:48.166 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.171+0000 7fe246765980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:48.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:48.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-05-09T20:28:48.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: starting shard=0 2024-05-09T20:28:48.167 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: starting shard=1 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: starting shard=2 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: starting shard=3 2024-05-09T20:28:48.168 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:48.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:48.169 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: starting shard=4 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 INFO: main: starting shard=5 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.175+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 INFO: main: starting shard=6 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:48.170 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:48.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 INFO: main: starting shard=7 2024-05-09T20:28:48.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:48.171 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 INFO: main: starting shard=8 2024-05-09T20:28:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:48.172 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 INFO: main: starting shard=9 2024-05-09T20:28:48.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:48.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.179+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:48.173 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 INFO: main: starting shard=10 2024-05-09T20:28:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 INFO: main: starting shard=11 2024-05-09T20:28:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:48.174 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:48.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 INFO: main: starting shard=12 2024-05-09T20:28:48.175 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:48.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:48.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.176 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.183+0000 7fe246765980 20 INFO: main: done 2024-05-09T20:28:48.178 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.187+0000 7fe0fa7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:48.182 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.191+0000 7fe246765980 20 remove_watcher() i=0 2024-05-09T20:28:48.183 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.191+0000 7fe246765980 2 removed watcher, disabling cache 2024-05-09T20:28:48.185 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.195+0000 7fe246765980 20 remove_watcher() i=1 2024-05-09T20:28:48.187 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.195+0000 7fe246765980 20 remove_watcher() i=2 2024-05-09T20:28:48.189 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.199+0000 7fe246765980 20 remove_watcher() i=3 2024-05-09T20:28:48.191 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.199+0000 7fe246765980 20 remove_watcher() i=4 2024-05-09T20:28:48.192 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.199+0000 7fe246765980 20 remove_watcher() i=5 2024-05-09T20:28:48.194 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.203+0000 7fe246765980 20 remove_watcher() i=6 2024-05-09T20:28:48.195 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.203+0000 7fe246765980 20 remove_watcher() i=7 2024-05-09T20:28:48.354 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:28:48.355 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket myver --object y 2024-05-09T20:28:48.392 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:48.392 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:48.436 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.443+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.438 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 realm 2024-05-09T20:28:48.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:48.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.439 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.440 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:48.440 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.447+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.451+0000 7f1a4e7da980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:48.441 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.451+0000 7f1a4e7da980 20 searching for the correct realm 2024-05-09T20:28:48.445 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.455+0000 7f1a4c2ee640 0 -- 172.21.15.63:0/4074156612 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f1a44012b50 msgr2=0x7f1a44078540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:48.664 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.671+0000 7f1a4c2ee640 0 -- 172.21.15.63:0/4074156612 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f1a44012b50 msgr2=0x7f1a44078540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:48.675 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:48.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:48.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:48.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:48.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:48.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:48.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.676 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.683+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:48.677 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 zone default found 2024-05-09T20:28:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 4 Realm: () 2024-05-09T20:28:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 zonegroup default 2024-05-09T20:28:48.678 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.679 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:48.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 started sync module instance, tier type = 2024-05-09T20:28:48.680 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.687+0000 7f1a4e7da980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:48.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.711+0000 7f1a4e7da980 20 add_watcher() i=0 2024-05-09T20:28:48.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.711+0000 7f1a4e7da980 20 add_watcher() i=1 2024-05-09T20:28:48.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.711+0000 7f1a4e7da980 20 add_watcher() i=2 2024-05-09T20:28:48.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.711+0000 7f1a4e7da980 20 add_watcher() i=3 2024-05-09T20:28:48.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.711+0000 7f1a4e7da980 20 add_watcher() i=4 2024-05-09T20:28:48.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.711+0000 7f1a4e7da980 20 add_watcher() i=5 2024-05-09T20:28:48.704 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.711+0000 7f1a4e7da980 20 add_watcher() i=6 2024-05-09T20:28:48.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.715+0000 7f1a4e7da980 20 add_watcher() i=7 2024-05-09T20:28:48.705 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.715+0000 7f1a4e7da980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:48.709 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.715+0000 7f19027e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:48.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:48.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:48.710 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 5 note: GC not initialized 2024-05-09T20:28:48.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f19017e2640 20 reqs_thread_entry: start 2024-05-09T20:28:48.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:48.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 Filter name: none 2024-05-09T20:28:48.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1872ffd640 20 reqs_thread_entry: start 2024-05-09T20:28:48.711 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:48.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 rados->read ofs=0 len=0 2024-05-09T20:28:48.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:48.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:48.712 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-05-09T20:28:48.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 INFO: main: starting shard=0 2024-05-09T20:28:48.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:48.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.719+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:48.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: starting shard=1 2024-05-09T20:28:48.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:48.713 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:48.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: starting shard=2 2024-05-09T20:28:48.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:48.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:48.714 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: starting shard=3 2024-05-09T20:28:48.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:48.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:48.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: starting shard=4 2024-05-09T20:28:48.715 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:48.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:48.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 INFO: main: starting shard=5 2024-05-09T20:28:48.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:48.716 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.723+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:48.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 INFO: main: starting shard=6 2024-05-09T20:28:48.717 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:48.718 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:48.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 INFO: main: starting shard=7 2024-05-09T20:28:48.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:48.719 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:48.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 INFO: main: starting shard=8 2024-05-09T20:28:48.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:48.720 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:48.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 INFO: main: starting shard=9 2024-05-09T20:28:48.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:48.721 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.727+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:48.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-09T20:28:48.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 INFO: main: starting shard=10 2024-05-09T20:28:48.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:48.722 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:48.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 INFO: main: starting shard=11 2024-05-09T20:28:48.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:48.723 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:48.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 INFO: main: starting shard=12 2024-05-09T20:28:48.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:48.724 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.731+0000 7f1a4e7da980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:48.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.735+0000 7f1a4e7da980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:48.725 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.735+0000 7f1a4e7da980 20 INFO: main: done 2024-05-09T20:28:48.727 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.735+0000 7f19027e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:48.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.739+0000 7f1a4e7da980 20 remove_watcher() i=0 2024-05-09T20:28:48.732 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.739+0000 7f1a4e7da980 2 removed watcher, disabling cache 2024-05-09T20:28:48.734 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.743+0000 7f1a4e7da980 20 remove_watcher() i=1 2024-05-09T20:28:48.737 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.743+0000 7f1a4e7da980 20 remove_watcher() i=2 2024-05-09T20:28:48.739 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.747+0000 7f1a4e7da980 20 remove_watcher() i=3 2024-05-09T20:28:48.742 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.751+0000 7f1a4e7da980 20 remove_watcher() i=4 2024-05-09T20:28:48.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.751+0000 7f1a4e7da980 20 remove_watcher() i=5 2024-05-09T20:28:48.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.755+0000 7f1a4e7da980 20 remove_watcher() i=6 2024-05-09T20:28:48.747 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:48.755+0000 7f1a4e7da980 20 remove_watcher() i=7 2024-05-09T20:28:48.926 INFO:tasks.workunit.client.0.smithi063.stderr:An error occurred (NoSuchKey) when calling the PutObject operation: None 2024-05-09T20:28:48.927 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bi list --bucket myver --object z 2024-05-09T20:28:48.963 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:48.963 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:49.008 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.015+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:49.010 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 realm 2024-05-09T20:28:49.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:49.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:49.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:49.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:49.012 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.019+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.023+0000 7f6c85b4f980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:49.013 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.023+0000 7f6c85b4f980 20 searching for the correct realm 2024-05-09T20:28:49.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.023+0000 7f6c83e64640 0 -- 172.21.15.63:0/4207364900 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f6c6c06a040 msgr2=0x7f6c6c04e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:49.229 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.235+0000 7f6c82e62640 0 -- 172.21.15.63:0/4207364900 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f6c740da430 msgr2=0x7f6c74110050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:49.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:49.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:49.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:49.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:49.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:49.244 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:49.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:49.245 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.251+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:49.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.246 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 zone default found 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 4 Realm: () 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 zonegroup default 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:49.247 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:49.248 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.255+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.259+0000 7f6c85b4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:49.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.259+0000 7f6c85b4f980 20 started sync module instance, tier type = 2024-05-09T20:28:49.249 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.259+0000 7f6c85b4f980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:49.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 20 add_watcher() i=0 2024-05-09T20:28:49.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 20 add_watcher() i=1 2024-05-09T20:28:49.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 20 add_watcher() i=2 2024-05-09T20:28:49.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 20 add_watcher() i=3 2024-05-09T20:28:49.273 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 20 add_watcher() i=4 2024-05-09T20:28:49.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 20 add_watcher() i=5 2024-05-09T20:28:49.274 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 20 add_watcher() i=6 2024-05-09T20:28:49.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 20 add_watcher() i=7 2024-05-09T20:28:49.276 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.283+0000 7f6c85b4f980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:49.281 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.287+0000 7f6b39fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:49.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:49.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:49.282 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 5 note: GC not initialized 2024-05-09T20:28:49.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6b38fe1640 20 reqs_thread_entry: start 2024-05-09T20:28:49.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:49.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 Filter name: none 2024-05-09T20:28:49.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6ab27fc640 20 reqs_thread_entry: start 2024-05-09T20:28:49.283 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:49.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 rados->read ofs=0 len=0 2024-05-09T20:28:49.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:49.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:49.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 INFO: main: max_entries=1000, index=gen=2, max_shards=13 2024-05-09T20:28:49.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 INFO: main: starting shard=0 2024-05-09T20:28:49.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:49.285 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.291+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:49.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=1, is_truncated=0, marker= 2024-05-09T20:28:49.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: starting shard=1 2024-05-09T20:28:49.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:49.286 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:49.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: starting shard=2 2024-05-09T20:28:49.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:49.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:49.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.287 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: starting shard=3 2024-05-09T20:28:49.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:49.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:49.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: starting shard=4 2024-05-09T20:28:49.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:49.288 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:49.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 INFO: main: starting shard=5 2024-05-09T20:28:49.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:49.289 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.295+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:49.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 INFO: main: starting shard=6 2024-05-09T20:28:49.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:49.290 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:49.291 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 INFO: main: starting shard=7 2024-05-09T20:28:49.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:49.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:49.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 INFO: main: starting shard=8 2024-05-09T20:28:49.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:49.292 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.299+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:49.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 INFO: main: starting shard=9 2024-05-09T20:28:49.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:49.294 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:49.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 INFO: main: starting shard=10 2024-05-09T20:28:49.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:49.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:49.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 INFO: main: starting shard=11 2024-05-09T20:28:49.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:49.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:49.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 INFO: main: starting shard=12 2024-05-09T20:28:49.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:49.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.303+0000 7f6c85b4f980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:49.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.307+0000 7f6c85b4f980 20 INFO: main: bi_list() returned without error; entries.size()=0, is_truncated=0, marker= 2024-05-09T20:28:49.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.307+0000 7f6c85b4f980 20 INFO: main: done 2024-05-09T20:28:49.300 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.307+0000 7f6b39fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:49.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.315+0000 7f6c85b4f980 20 remove_watcher() i=0 2024-05-09T20:28:49.305 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.315+0000 7f6c85b4f980 2 removed watcher, disabling cache 2024-05-09T20:28:49.307 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.315+0000 7f6c85b4f980 20 remove_watcher() i=1 2024-05-09T20:28:49.310 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.319+0000 7f6c85b4f980 20 remove_watcher() i=2 2024-05-09T20:28:49.312 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.319+0000 7f6c85b4f980 20 remove_watcher() i=3 2024-05-09T20:28:49.314 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.323+0000 7f6c85b4f980 20 remove_watcher() i=4 2024-05-09T20:28:49.315 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.323+0000 7f6c85b4f980 20 remove_watcher() i=5 2024-05-09T20:28:49.317 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.327+0000 7f6c85b4f980 20 remove_watcher() i=6 2024-05-09T20:28:49.319 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:49.327+0000 7f6c85b4f980 20 remove_watcher() i=7 2024-05-09T20:28:49.367 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config rm client rgw_debug_inject_set_olh_err 2024-05-09T20:28:49.791 INFO:tasks.workunit.client.0.smithi063.stderr:ceph config rm client rgw_debug_inject_olh_cancel_modification_err 2024-05-09T20:28:50.224 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket reshard --bucket myver --num-shards 17 --yes-i-really-mean-it 2024-05-09T20:28:50.252 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:50.252 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:50.293 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.295 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 20 realm 2024-05-09T20:28:50.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:50.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.296 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.303+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.307+0000 7f95667ce980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:50.297 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.307+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.307+0000 7f95667ce980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:50.298 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.307+0000 7f95667ce980 20 searching for the correct realm 2024-05-09T20:28:50.302 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.311+0000 7f95642e2640 0 -- 172.21.15.63:0/3518040035 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f955403bc10 msgr2=0x7f9554030920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:50.520 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.527+0000 7f95642e2640 0 -- 172.21.15.63:0/3518040035 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f955403bc10 msgr2=0x7f9554030920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:50.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:50.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:50.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:50.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:50.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:50.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:50.532 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.539+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.533 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:50.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:50.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 zone default found 2024-05-09T20:28:50.534 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 4 Realm: () 2024-05-09T20:28:50.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:50.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:50.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:50.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 zonegroup default 2024-05-09T20:28:50.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.535 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.536 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 started sync module instance, tier type = 2024-05-09T20:28:50.537 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.543+0000 7f95667ce980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:50.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 20 add_watcher() i=0 2024-05-09T20:28:50.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 20 add_watcher() i=1 2024-05-09T20:28:50.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 20 add_watcher() i=2 2024-05-09T20:28:50.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 20 add_watcher() i=3 2024-05-09T20:28:50.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 20 add_watcher() i=4 2024-05-09T20:28:50.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 20 add_watcher() i=5 2024-05-09T20:28:50.562 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 20 add_watcher() i=6 2024-05-09T20:28:50.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 20 add_watcher() i=7 2024-05-09T20:28:50.564 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.571+0000 7f95667ce980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:50.568 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.575+0000 7f941afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:50.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:50.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:50.569 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 5 note: GC not initialized 2024-05-09T20:28:50.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f9419fdb640 20 reqs_thread_entry: start 2024-05-09T20:28:50.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:50.570 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 20 Filter name: none 2024-05-09T20:28:50.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f938b7fe640 20 reqs_thread_entry: start 2024-05-09T20:28:50.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-05-09T20:28:50.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:50.571 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-05-09T20:28:50.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-05-09T20:28:50.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 : miss 2024-05-09T20:28:50.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:50.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:50.572 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 adding default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:50.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:50.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:50.573 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.579+0000 7f95667ce980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 2024-05-09T20:28:50.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f95667ce980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:50.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f95667ce980 10 moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:50.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f95667ce980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:50.585 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f95667ce980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18, ofs0, ns] 2024-05-09T20:28:50.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f943affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431611 cookie 94273590266736 notifier 6890 bl.length()=813 2024-05-09T20:28:50.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f943affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:50.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f943affd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:50.586 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f943affd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:50.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f95667ce980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_sources= new_sources= 2024-05-09T20:28:50.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f95667ce980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential sources added=[] removed=[] 2024-05-09T20:28:50.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f95667ce980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_dests= new_dests= 2024-05-09T20:28:50.588 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.595+0000 7f95667ce980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential dests added=[] removed=[] 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.0 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.0 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.1 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.1 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.2 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.2 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.3 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.3 2024-05-09T20:28:50.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.4 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.4 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.5 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.5 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.6 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.6 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.7 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.7 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.8 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.8 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.10 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.10 2024-05-09T20:28:50.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.11 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.11 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.12 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.12 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.13 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.13 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.14 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.14 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.15 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.15 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.16 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.16 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:50.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.0 2024-05-09T20:28:50.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:50.598 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.1 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.2 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.603+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.3 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.607+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.607+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.4 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.607+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.607+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.5 2024-05-09T20:28:50.599 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.607+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:50.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.607+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.6 2024-05-09T20:28:50.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.607+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:50.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.607+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.7 2024-05-09T20:28:50.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.611+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:50.601 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.611+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.8 2024-05-09T20:28:50.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.611+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:50.602 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.611+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.9 2024-05-09T20:28:50.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.611+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:50.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.611+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.10 2024-05-09T20:28:50.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.611+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:50.604 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.611+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.11 2024-05-09T20:28:50.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.615+0000 7f95667ce980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:50.605 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.615+0000 7f95667ce980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.2.12 2024-05-09T20:28:50.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.619+0000 7f95667ce980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 : hit (requested=0x17, cached=0x17) 2024-05-09T20:28:50.611 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.619+0000 7f95667ce980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 2024-05-09T20:28:50.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f95667ce980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:50.613 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f95667ce980 10 moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:50.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f95667ce980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:50.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f95667ce980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18, ofs0, ns] 2024-05-09T20:28:50.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f943b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431612 cookie 94273590266736 notifier 6890 bl.length()=781 2024-05-09T20:28:50.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f943b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:50.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f943b7fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:50.614 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f943b7fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:50.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f95667ce980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_sources= new_sources= 2024-05-09T20:28:50.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f95667ce980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential sources added=[] removed=[] 2024-05-09T20:28:50.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f95667ce980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): orig_dests= new_dests= 2024-05-09T20:28:50.617 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.623+0000 7f95667ce980 20 handle_bi_update(): bucket=:myver[6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18]): potential dests added=[] removed=[] 2024-05-09T20:28:50.622 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.631+0000 7f95667ce980 1 execute INFO: reshard of bucket "myver" completed successfully 2024-05-09T20:28:50.626 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.635+0000 7f941afdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:50.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.639+0000 7f95667ce980 20 remove_watcher() i=0 2024-05-09T20:28:50.630 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.639+0000 7f95667ce980 2 removed watcher, disabling cache 2024-05-09T20:28:50.633 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.639+0000 7f95667ce980 20 remove_watcher() i=1 2024-05-09T20:28:50.635 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.643+0000 7f95667ce980 20 remove_watcher() i=2 2024-05-09T20:28:50.638 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.647+0000 7f95667ce980 20 remove_watcher() i=3 2024-05-09T20:28:50.640 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.647+0000 7f95667ce980 20 remove_watcher() i=4 2024-05-09T20:28:50.641 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.651+0000 7f95667ce980 20 remove_watcher() i=5 2024-05-09T20:28:50.643 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.651+0000 7f95667ce980 20 remove_watcher() i=6 2024-05-09T20:28:50.644 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.651+0000 7f95667ce980 20 remove_watcher() i=7 2024-05-09T20:28:50.663 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket check unlinked --bucket myver --fix --min-age-hours 0 --rgw-olh-pending-timeout-sec 0 --dump-keys 2024-05-09T20:28:50.700 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:50.701 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:50.744 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.751+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 realm 2024-05-09T20:28:50.745 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:50.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.746 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:50.748 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:50.749 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.755+0000 7fadbae85980 20 searching for the correct realm 2024-05-09T20:28:50.752 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.759+0000 7fadb8999640 0 -- 172.21.15.63:0/1064766120 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fadb0018040 msgr2=0x7fadb0071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:50.967 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.975+0000 7fadb8999640 0 -- 172.21.15.63:0/1064766120 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fadb0018040 msgr2=0x7fadb0071040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:50.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:50.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:50.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:50.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:50.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:50.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:50.980 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.981 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.987+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:50.982 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 zone default found 2024-05-09T20:28:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 4 Realm: () 2024-05-09T20:28:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 zonegroup default 2024-05-09T20:28:50.983 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.984 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:50.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:50.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 started sync module instance, tier type = 2024-05-09T20:28:50.985 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:50.991+0000 7fadbae85980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:51.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.015+0000 7fadbae85980 20 add_watcher() i=0 2024-05-09T20:28:51.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.015+0000 7fadbae85980 20 add_watcher() i=1 2024-05-09T20:28:51.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.015+0000 7fadbae85980 20 add_watcher() i=2 2024-05-09T20:28:51.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.015+0000 7fadbae85980 20 add_watcher() i=3 2024-05-09T20:28:51.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.015+0000 7fadbae85980 20 add_watcher() i=4 2024-05-09T20:28:51.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.015+0000 7fadbae85980 20 add_watcher() i=5 2024-05-09T20:28:51.009 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.015+0000 7fadbae85980 20 add_watcher() i=6 2024-05-09T20:28:51.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.019+0000 7fadbae85980 20 add_watcher() i=7 2024-05-09T20:28:51.011 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.019+0000 7fadbae85980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:51.015 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.023+0000 7fac6f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:51.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.023+0000 7fadbae85980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:51.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.023+0000 7fadbae85980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:51.016 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.023+0000 7fadbae85980 5 note: GC not initialized 2024-05-09T20:28:51.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.023+0000 7fac6e7e4640 20 reqs_thread_entry: start 2024-05-09T20:28:51.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:51.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 20 Filter name: none 2024-05-09T20:28:51.017 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fac67fff640 20 reqs_thread_entry: start 2024-05-09T20:28:51.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 cache get: name=default.rgw.meta+root+myver : miss 2024-05-09T20:28:51.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:51.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 cache put: name=default.rgw.meta+root+myver info.flags=0x11 2024-05-09T20:28:51.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 adding default.rgw.meta+root+myver to cache LRU end 2024-05-09T20:28:51.018 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 cache get: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 : miss 2024-05-09T20:28:51.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.019 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:51.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 cache put: name=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 info.flags=0x17 2024-05-09T20:28:51.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 adding default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 to cache LRU end 2024-05-09T20:28:51.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:28:51.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 updating xattr: name=user.rgw.acl bl.length()=143 2024-05-09T20:28:51.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myver:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18 2024-05-09T20:28:51.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.0 2024-05-09T20:28:51.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.0 2024-05-09T20:28:51.020 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 1 NOTICE: finished shard 0 (0 entries removed) 2024-05-09T20:28:51.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.1 2024-05-09T20:28:51.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.027+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.1 2024-05-09T20:28:51.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 1 NOTICE: finished shard 1 (0 entries removed) 2024-05-09T20:28:51.021 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.2 2024-05-09T20:28:51.022 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.2 2024-05-09T20:28:51.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 1 NOTICE: finished shard 2 (0 entries removed) 2024-05-09T20:28:51.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.3 2024-05-09T20:28:51.023 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.3 2024-05-09T20:28:51.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 1 NOTICE: finished shard 3 (0 entries removed) 2024-05-09T20:28:51.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.4 2024-05-09T20:28:51.024 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.031+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.4 2024-05-09T20:28:51.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 1 NOTICE: finished shard 4 (0 entries removed) 2024-05-09T20:28:51.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.5 2024-05-09T20:28:51.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.5 2024-05-09T20:28:51.025 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 1 NOTICE: finished shard 5 (0 entries removed) 2024-05-09T20:28:51.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.6 2024-05-09T20:28:51.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.6 2024-05-09T20:28:51.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 1 NOTICE: finished shard 6 (0 entries removed) 2024-05-09T20:28:51.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.7 2024-05-09T20:28:51.026 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.7 2024-05-09T20:28:51.027 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 1 NOTICE: finished shard 7 (0 entries removed) 2024-05-09T20:28:51.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.8 2024-05-09T20:28:51.028 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.8 2024-05-09T20:28:51.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 1 NOTICE: finished shard 8 (0 entries removed) 2024-05-09T20:28:51.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.029 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.035+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.039+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.10 2024-05-09T20:28:51.030 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.039+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.10 2024-05-09T20:28:51.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.039+0000 7fadbae85980 1 NOTICE: finished shard 10 (0 entries removed) 2024-05-09T20:28:51.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.039+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.11 2024-05-09T20:28:51.032 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.039+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.11 2024-05-09T20:28:51.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 1 NOTICE: finished shard 11 (0 entries removed) 2024-05-09T20:28:51.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.12 2024-05-09T20:28:51.033 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.12 2024-05-09T20:28:51.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 1 NOTICE: finished shard 12 (0 entries removed) 2024-05-09T20:28:51.034 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.13 2024-05-09T20:28:51.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.13 2024-05-09T20:28:51.035 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 1 NOTICE: finished shard 13 (0 entries removed) 2024-05-09T20:28:51.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.14 2024-05-09T20:28:51.036 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.14 2024-05-09T20:28:51.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 1 NOTICE: finished shard 14 (0 entries removed) 2024-05-09T20:28:51.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.15 2024-05-09T20:28:51.037 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.043+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.15 2024-05-09T20:28:51.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.047+0000 7fadbae85980 1 NOTICE: finished shard 15 (0 entries removed) 2024-05-09T20:28:51.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.047+0000 7fadbae85980 10 bucket_obj is .dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.16 2024-05-09T20:28:51.038 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.047+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.16 2024-05-09T20:28:51.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.047+0000 7fadbae85980 1 NOTICE: finished shard 16 (0 entries removed) 2024-05-09T20:28:51.039 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.047+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.040 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.047+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:x state=0x55805e5f0dd8 s->prefetch_data=0 2024-05-09T20:28:51.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.051+0000 7fadbae85980 20 get_obj_state: setting s->obj_tag to i1cs0i21xnpapjn90l4sy6ndzhk6cfo4 2024-05-09T20:28:51.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.051+0000 7fadbae85980 20 get_obj_state_impl: setting s->olh_tag to y10mspey0i8z3jt2atqidw5f41wskpfx 2024-05-09T20:28:51.041 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.051+0000 7fadbae85980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=y10mspey0i8z3jt2atqidw5f41wskpfx 2024-05-09T20:28:51.043 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.051+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.046 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 20 olh_log_entry: epoch=2 op=2 key=x[] 2024-05-09T20:28:51.047 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 20 olh_log_entry: epoch=2 op=3 key=x[j-nZ.VXZS2uvIeZcUDa1link6djBV0h] 2024-05-09T20:28:51.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:_:j-nZ.VXZS2uvIeZcUDa1link6djBV0h_x state=0x55805e5f42e8 s->prefetch_data=0 2024-05-09T20:28:51.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 10 manifest: total_size = 9 2024-05-09T20:28:51.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8015693094854616953 2024-05-09T20:28:51.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:_:j-nZ.VXZS2uvIeZcUDa1link6djBV0h_x state=0x55805e5f42e8 s->prefetch_data=0 2024-05-09T20:28:51.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 20 prepare_atomic_modification: state is not atomic. state=0x55805e5f42e8 2024-05-09T20:28:51.048 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.055+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.063+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:x state=0x55805e5f0dd8 s->prefetch_data=0 2024-05-09T20:28:51.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.063+0000 7fadbae85980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=y10mspey0i8z3jt2atqidw5f41wskpfx 2024-05-09T20:28:51.057 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.067+0000 7fac8f7fe640 20 handle_completion(): completion ok for obj=x[j-nZ.VXZS2uvIeZcUDa1link6djBV0h] 2024-05-09T20:28:51.062 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.071+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.065 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.071+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.066 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.075+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.075+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:y state=0x55805e5f0dd8 s->prefetch_data=0 2024-05-09T20:28:51.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.075+0000 7fadbae85980 20 get_obj_state: setting s->obj_tag to 8g1m9jxcmcv1y0un6d1t26hucnb6wity 2024-05-09T20:28:51.067 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.075+0000 7fadbae85980 20 get_obj_state_impl: setting s->olh_tag to eo5lcivvmcf1m3t0ju14u2eadblporez 2024-05-09T20:28:51.068 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.075+0000 7fadbae85980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=eo5lcivvmcf1m3t0ju14u2eadblporez 2024-05-09T20:28:51.071 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.079+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.074 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 20 olh_log_entry: epoch=2 op=2 key=y[] 2024-05-09T20:28:51.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 20 olh_log_entry: epoch=2 op=3 key=y[QHwBxKnnBv3E5rZG0l8qtcxHvXRha.m] 2024-05-09T20:28:51.075 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:_:QHwBxKnnBv3E5rZG0l8qtcxHvXRha.m_y state=0x55805e5f42e8 s->prefetch_data=0 2024-05-09T20:28:51.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 10 manifest: total_size = 9 2024-05-09T20:28:51.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12320127199655635855 2024-05-09T20:28:51.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:_:QHwBxKnnBv3E5rZG0l8qtcxHvXRha.m_y state=0x55805e5f42e8 s->prefetch_data=0 2024-05-09T20:28:51.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 20 prepare_atomic_modification: state is not atomic. state=0x55805e5f42e8 2024-05-09T20:28:51.076 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.083+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.085 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.095+0000 7fac8f7fe640 20 handle_completion(): completion ok for obj=y[QHwBxKnnBv3E5rZG0l8qtcxHvXRha.m] 2024-05-09T20:28:51.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.095+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:y state=0x55805e5f0dd8 s->prefetch_data=0 2024-05-09T20:28:51.086 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.095+0000 7fadbae85980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=eo5lcivvmcf1m3t0ju14u2eadblporez 2024-05-09T20:28:51.094 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.103+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.097 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.107+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.098 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.107+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.107+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:z state=0x55805e5f0dd8 s->prefetch_data=0 2024-05-09T20:28:51.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.107+0000 7fadbae85980 20 get_obj_state: setting s->obj_tag to ausasr9u1nfg6bmuqactcolghal4qhb1 2024-05-09T20:28:51.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.107+0000 7fadbae85980 20 get_obj_state_impl: setting s->olh_tag to chgfag8xap3gx85v942go07w5njl5r47 2024-05-09T20:28:51.099 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.107+0000 7fadbae85980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=chgfag8xap3gx85v942go07w5njl5r47 2024-05-09T20:28:51.102 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.111+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.105 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 20 olh_log_entry: epoch=2 op=2 key=z[] 2024-05-09T20:28:51.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 20 olh_log_entry: epoch=2 op=3 key=z[wpLdkg-O-nl6MhZ9Tg1qWebBrjLB1L0] 2024-05-09T20:28:51.106 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:_:wpLdkg-O-nl6MhZ9Tg1qWebBrjLB1L0_z state=0x55805e62e0b8 s->prefetch_data=0 2024-05-09T20:28:51.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 10 manifest: total_size = 11 2024-05-09T20:28:51.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 20 get_obj_state: setting s->obj_tag to 6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12604467338605100704 2024-05-09T20:28:51.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:_:wpLdkg-O-nl6MhZ9Tg1qWebBrjLB1L0_z state=0x55805e62e0b8 s->prefetch_data=0 2024-05-09T20:28:51.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 20 prepare_atomic_modification: state is not atomic. state=0x55805e62e0b8 2024-05-09T20:28:51.107 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.115+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.123+0000 7fac8ffff640 20 handle_completion(): completion ok for obj=z[wpLdkg-O-nl6MhZ9Tg1qWebBrjLB1L0] 2024-05-09T20:28:51.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.123+0000 7fadbae85980 20 get_obj_state: octx=0x55805e340f60 obj=myver:z state=0x55805e5f0dd8 s->prefetch_data=0 2024-05-09T20:28:51.116 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.123+0000 7fadbae85980 20 bucket_index_guard_olh_op(): olh_state.olh_tag=chgfag8xap3gx85v942go07w5njl5r47 2024-05-09T20:28:51.122 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.131+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.131+0000 7fadbae85980 20 bucket index object: :.dir.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18.3.9 2024-05-09T20:28:51.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.135+0000 7fadbae85980 1 NOTICE: finished shard 9 (3 entries removed) 2024-05-09T20:28:51.125 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.135+0000 7fadbae85980 1 NOTICE: finished all shards (3 entries removed) 2024-05-09T20:28:51.127 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.135+0000 7fac6f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:51.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.139+0000 7fadbae85980 20 remove_watcher() i=0 2024-05-09T20:28:51.132 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.139+0000 7fadbae85980 2 removed watcher, disabling cache 2024-05-09T20:28:51.134 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.143+0000 7fadbae85980 20 remove_watcher() i=1 2024-05-09T20:28:51.137 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.143+0000 7fadbae85980 20 remove_watcher() i=2 2024-05-09T20:28:51.139 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.147+0000 7fadbae85980 20 remove_watcher() i=3 2024-05-09T20:28:51.141 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.151+0000 7fadbae85980 20 remove_watcher() i=4 2024-05-09T20:28:51.143 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.151+0000 7fadbae85980 20 remove_watcher() i=5 2024-05-09T20:28:51.145 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.155+0000 7fadbae85980 20 remove_watcher() i=6 2024-05-09T20:28:51.148 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.155+0000 7fadbae85980 20 remove_watcher() i=7 2024-05-09T20:28:51.513 INFO:tasks.workunit.client.0.smithi063.stderr:radosgw-admin bucket stats --bucket myver 2024-05-09T20:28:51.547 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:28:51.547 INFO:tasks.workunit.client.0.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:28:51.592 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.599+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:51.594 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 realm 2024-05-09T20:28:51.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:51.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:28:51.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:51.595 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:51.596 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.603+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.607+0000 7f167b1e9980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:28:51.597 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.607+0000 7f167b1e9980 20 searching for the correct realm 2024-05-09T20:28:51.600 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.607+0000 7f16794fe640 0 -- 172.21.15.63:0/3214642640 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f166c032060 msgr2=0x7f166c05ae10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:28:51.815 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.823+0000 7f16794fe640 0 -- 172.21.15.63:0/3214642640 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f166c032060 msgr2=0x7f166c05ae10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:28:51.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:28:51.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:28:51.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:28:51.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:28:51.831 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:28:51.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:28:51.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:51.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:28:51.832 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.839+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:28:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 zone default found 2024-05-09T20:28:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 4 Realm: () 2024-05-09T20:28:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:28:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:28:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:28:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 zonegroup default 2024-05-09T20:28:51.833 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:51.834 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:51.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:28:51.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 started sync module instance, tier type = 2024-05-09T20:28:51.835 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.843+0000 7f167b1e9980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:28:51.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 20 add_watcher() i=0 2024-05-09T20:28:51.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 20 add_watcher() i=1 2024-05-09T20:28:51.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 20 add_watcher() i=2 2024-05-09T20:28:51.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 20 add_watcher() i=3 2024-05-09T20:28:51.857 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 20 add_watcher() i=4 2024-05-09T20:28:51.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 20 add_watcher() i=5 2024-05-09T20:28:51.858 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 20 add_watcher() i=6 2024-05-09T20:28:51.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 20 add_watcher() i=7 2024-05-09T20:28:51.859 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.867+0000 7f167b1e9980 2 all 8 watchers are set, enabling cache 2024-05-09T20:28:51.863 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.871+0000 7f152efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:51.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.871+0000 7f167b1e9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:28:51.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.871+0000 7f167b1e9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:28:51.864 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.871+0000 7f167b1e9980 5 note: GC not initialized 2024-05-09T20:28:51.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.871+0000 7f152dfe3640 20 reqs_thread_entry: start 2024-05-09T20:28:51.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.871+0000 7f167b1e9980 20 init_complete bucket index max shards: 11 2024-05-09T20:28:51.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.871+0000 7f167b1e9980 20 Filter name: none 2024-05-09T20:28:51.865 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f149f7fe640 20 reqs_thread_entry: start 2024-05-09T20:28:51.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:51.866 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:51.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:51.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 rados_obj.operate() r=0 bl.length=173 2024-05-09T20:28:51.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 rados->read ofs=0 len=0 2024-05-09T20:28:51.867 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 rados_obj.operate() r=0 bl.length=387 2024-05-09T20:28:51.868 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.875+0000 7f167b1e9980 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-09T20:28:51.872 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.879+0000 7f152efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:28:51.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.883+0000 7f167b1e9980 20 remove_watcher() i=0 2024-05-09T20:28:51.877 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.883+0000 7f167b1e9980 2 removed watcher, disabling cache 2024-05-09T20:28:51.879 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.887+0000 7f167b1e9980 20 remove_watcher() i=1 2024-05-09T20:28:51.881 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.891+0000 7f167b1e9980 20 remove_watcher() i=2 2024-05-09T20:28:51.883 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.891+0000 7f167b1e9980 20 remove_watcher() i=3 2024-05-09T20:28:51.885 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.895+0000 7f167b1e9980 20 remove_watcher() i=4 2024-05-09T20:28:51.887 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.895+0000 7f167b1e9980 20 remove_watcher() i=5 2024-05-09T20:28:51.888 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.895+0000 7f167b1e9980 20 remove_watcher() i=6 2024-05-09T20:28:51.890 INFO:tasks.workunit.client.0.smithi063.stderr:2024-05-09T20:28:51.899+0000 7f167b1e9980 20 remove_watcher() i=7 2024-05-09T20:28:51.914 INFO:tasks.workunit.client.0.smithi063.stderr:{'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}} 2024-05-09T20:28:51.914 INFO:tasks.workunit.client.0.smithi063.stderr:Deleting bucket a-bucket 2024-05-09T20:28:52.055 INFO:tasks.workunit.client.0.smithi063.stderr:Deleting bucket myver 2024-05-09T20:28:52.656 INFO:tasks.workunit.client.0.smithi063.stderr:Completed resharding tests 2024-05-09T20:28:52.656 INFO:tasks.workunit.client.0.smithi063.stdout:connected to http://localhost:80 2024-05-09T20:28:52.765 INFO:tasks.workunit.client.0.smithi063.stderr:+ deactivate 2024-05-09T20:28:52.765 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' -n /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin ']' 2024-05-09T20:28:52.765 INFO:tasks.workunit.client.0.smithi063.stderr:+ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2024-05-09T20:28:52.765 INFO:tasks.workunit.client.0.smithi063.stderr:+ export PATH 2024-05-09T20:28:52.765 INFO:tasks.workunit.client.0.smithi063.stderr:+ unset _OLD_VIRTUAL_PATH 2024-05-09T20:28:52.765 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' -n '' ']' 2024-05-09T20:28:52.766 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' -n /usr/bin/bash -o -n '' ']' 2024-05-09T20:28:52.766 INFO:tasks.workunit.client.0.smithi063.stderr:+ hash -r 2024-05-09T20:28:52.766 INFO:tasks.workunit.client.0.smithi063.stdout:OK. 2024-05-09T20:28:52.767 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' -n '' ']' 2024-05-09T20:28:52.767 INFO:tasks.workunit.client.0.smithi063.stderr:+ unset VIRTUAL_ENV 2024-05-09T20:28:52.767 INFO:tasks.workunit.client.0.smithi063.stderr:+ unset VIRTUAL_ENV_PROMPT 2024-05-09T20:28:52.767 INFO:tasks.workunit.client.0.smithi063.stderr:+ '[' '!' '' = nondestructive ']' 2024-05-09T20:28:52.767 INFO:tasks.workunit.client.0.smithi063.stderr:+ unset -f deactivate 2024-05-09T20:28:52.767 INFO:tasks.workunit.client.0.smithi063.stderr:+ echo OK. 2024-05-09T20:28:52.767 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T20:28:52.768 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T20:28:52.822 INFO:tasks.workunit:Stopping ['rgw/run-reshard.sh'] on client.0... 2024-05-09T20:28:52.822 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T20:28:53.280 DEBUG:teuthology.parallel:result is None 2024-05-09T20:28:53.281 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:28:53.293 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:28:53.293 DEBUG:teuthology.orchestra.run.smithi063:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-09T20:28:53.341 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T20:28:53.341 INFO:teuthology.run_tasks:Running task s3tests-java... 2024-05-09T20:28:53.355 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2024-05-09T20:28:53.355 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2024-05-09T20:28:53.355 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2024-05-09T20:28:53.356 DEBUG:teuthology.orchestra.run.smithi063:> git clone -b ceph-master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2024-05-09T20:28:53.394 INFO:teuthology.orchestra.run.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2024-05-09T20:28:58.126 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2024-05-09T20:28:58.126 DEBUG:teuthology.orchestra.run.smithi063:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2024-05-09T20:28:58.218 INFO:teuthology.orchestra.run.smithi063.stderr:/home/ubuntu/cephtest/s3-tests-java/bootstrap.sh: missing required DEB packages. Installing via sudo. 2024-05-09T20:28:58.279 INFO:teuthology.orchestra.run.smithi063.stdout:Reading package lists... 2024-05-09T20:28:58.488 INFO:teuthology.orchestra.run.smithi063.stdout:Building dependency tree... 2024-05-09T20:28:58.489 INFO:teuthology.orchestra.run.smithi063.stdout:Reading state information... 2024-05-09T20:28:58.695 INFO:teuthology.orchestra.run.smithi063.stdout:The following packages were automatically installed and are no longer required: 2024-05-09T20:28:58.695 INFO:teuthology.orchestra.run.smithi063.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-05-09T20:28:58.695 INFO:teuthology.orchestra.run.smithi063.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-05-09T20:28:58.695 INFO:teuthology.orchestra.run.smithi063.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-05-09T20:28:58.696 INFO:teuthology.orchestra.run.smithi063.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-05-09T20:28:58.696 INFO:teuthology.orchestra.run.smithi063.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-05-09T20:28:58.696 INFO:teuthology.orchestra.run.smithi063.stdout:Use 'sudo apt autoremove' to remove them. 2024-05-09T20:28:58.697 INFO:teuthology.orchestra.run.smithi063.stdout:The following additional packages will be installed: 2024-05-09T20:28:58.697 INFO:teuthology.orchestra.run.smithi063.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-05-09T20:28:58.698 INFO:teuthology.orchestra.run.smithi063.stdout: openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-05-09T20:28:58.699 INFO:teuthology.orchestra.run.smithi063.stdout:Suggested packages: 2024-05-09T20:28:58.699 INFO:teuthology.orchestra.run.smithi063.stdout: gvfs openjdk-8-demo openjdk-8-source visualvm libnss-mdns fonts-nanum 2024-05-09T20:28:58.699 INFO:teuthology.orchestra.run.smithi063.stdout: fonts-ipafont-gothic fonts-ipafont-mincho fonts-wqy-microhei 2024-05-09T20:28:58.699 INFO:teuthology.orchestra.run.smithi063.stdout: fonts-wqy-zenhei fonts-indic 2024-05-09T20:28:58.732 INFO:teuthology.orchestra.run.smithi063.stdout:The following NEW packages will be installed: 2024-05-09T20:28:58.733 INFO:teuthology.orchestra.run.smithi063.stdout: libgail-common libgail18 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common 2024-05-09T20:28:58.733 INFO:teuthology.orchestra.run.smithi063.stdout: openjdk-8-jdk openjdk-8-jdk-headless openjdk-8-jre openjdk-8-jre-headless 2024-05-09T20:28:58.796 INFO:teuthology.orchestra.run.smithi063.stdout:0 upgraded, 9 newly installed, 0 to remove and 307 not upgraded. 2024-05-09T20:28:58.796 INFO:teuthology.orchestra.run.smithi063.stdout:Need to get 46.1 MB of archives. 2024-05-09T20:28:58.796 INFO:teuthology.orchestra.run.smithi063.stdout:After this operation, 155 MB of additional disk space will be used. 2024-05-09T20:28:58.796 INFO:teuthology.orchestra.run.smithi063.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-common all 2.24.33-2ubuntu2 [125 kB] 2024-05-09T20:28:58.888 INFO:teuthology.orchestra.run.smithi063.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-0 amd64 2.24.33-2ubuntu2 [2,037 kB] 2024-05-09T20:28:58.987 INFO:teuthology.orchestra.run.smithi063.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail18 amd64 2.24.33-2ubuntu2 [15.9 kB] 2024-05-09T20:28:58.987 INFO:teuthology.orchestra.run.smithi063.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgail-common amd64 2.24.33-2ubuntu2 [132 kB] 2024-05-09T20:28:58.991 INFO:teuthology.orchestra.run.smithi063.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgtk2.0-bin amd64 2.24.33-2ubuntu2 [7,932 B] 2024-05-09T20:28:58.991 INFO:teuthology.orchestra.run.smithi063.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre-headless amd64 8u402-ga-2ubuntu1~22.04 [30.8 MB] 2024-05-09T20:28:59.430 INFO:teuthology.orchestra.run.smithi063.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jre amd64 8u402-ga-2ubuntu1~22.04 [75.5 kB] 2024-05-09T20:28:59.431 INFO:teuthology.orchestra.run.smithi063.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk-headless amd64 8u402-ga-2ubuntu1~22.04 [8,873 kB] 2024-05-09T20:28:59.519 INFO:teuthology.orchestra.run.smithi063.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 openjdk-8-jdk amd64 8u402-ga-2ubuntu1~22.04 [4,068 kB] 2024-05-09T20:28:59.866 INFO:teuthology.orchestra.run.smithi063.stderr:debconf: unable to initialize frontend: Dialog 2024-05-09T20:28:59.866 INFO:teuthology.orchestra.run.smithi063.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-05-09T20:28:59.866 INFO:teuthology.orchestra.run.smithi063.stderr:debconf: falling back to frontend: Readline 2024-05-09T20:28:59.883 INFO:teuthology.orchestra.run.smithi063.stderr:debconf: unable to initialize frontend: Readline 2024-05-09T20:28:59.883 INFO:teuthology.orchestra.run.smithi063.stderr:debconf: (This frontend requires a controlling tty.) 2024-05-09T20:28:59.883 INFO:teuthology.orchestra.run.smithi063.stderr:debconf: falling back to frontend: Teletype 2024-05-09T20:28:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-05-09T20:28:59.923 INFO:teuthology.orchestra.run.smithi063.stdout:Fetched 46.1 MB in 1s (57.1 MB/s) 2024-05-09T20:29:00.040 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libgtk2.0-common. 2024-05-09T20:29:00.103 INFO:teuthology.orchestra.run.smithi063.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174892 files and directories currently installed.) 2024-05-09T20:29:00.108 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../0-libgtk2.0-common_2.24.33-2ubuntu2_all.deb ... 2024-05-09T20:29:00.147 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-05-09T20:29:00.535 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libgtk2.0-0:amd64. 2024-05-09T20:29:00.564 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../1-libgtk2.0-0_2.24.33-2ubuntu2_amd64.deb ... 2024-05-09T20:29:00.643 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-05-09T20:29:01.090 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libgail18:amd64. 2024-05-09T20:29:01.119 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../2-libgail18_2.24.33-2ubuntu2_amd64.deb ... 2024-05-09T20:29:01.156 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-05-09T20:29:01.409 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libgail-common:amd64. 2024-05-09T20:29:01.438 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../3-libgail-common_2.24.33-2ubuntu2_amd64.deb ... 2024-05-09T20:29:01.475 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-05-09T20:29:01.745 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package libgtk2.0-bin. 2024-05-09T20:29:01.762 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../4-libgtk2.0-bin_2.24.33-2ubuntu2_amd64.deb ... 2024-05-09T20:29:01.803 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-05-09T20:29:02.332 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package openjdk-8-jre-headless:amd64. 2024-05-09T20:29:02.360 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../5-openjdk-8-jre-headless_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-05-09T20:29:02.402 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking openjdk-8-jre-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-09T20:29:03.954 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package openjdk-8-jre:amd64. 2024-05-09T20:29:03.983 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../6-openjdk-8-jre_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-05-09T20:29:04.025 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking openjdk-8-jre:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-09T20:29:04.534 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package openjdk-8-jdk-headless:amd64. 2024-05-09T20:29:04.563 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../7-openjdk-8-jdk-headless_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-05-09T20:29:04.605 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking openjdk-8-jdk-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-09T20:29:05.380 INFO:teuthology.orchestra.run.smithi063.stdout:Selecting previously unselected package openjdk-8-jdk:amd64. 2024-05-09T20:29:05.409 INFO:teuthology.orchestra.run.smithi063.stdout:Preparing to unpack .../8-openjdk-8-jdk_8u402-ga-2ubuntu1~22.04_amd64.deb ... 2024-05-09T20:29:05.441 INFO:teuthology.orchestra.run.smithi063.stdout:Unpacking openjdk-8-jdk:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-09T20:29:05.784 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up openjdk-8-jre-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-09T20:29:06.955 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/orbd to provide /usr/bin/orbd (orbd) in auto mode 2024-05-09T20:29:06.981 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/servertool to provide /usr/bin/servertool (servertool) in auto mode 2024-05-09T20:29:07.006 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/tnameserv to provide /usr/bin/tnameserv (tnameserv) in auto mode 2024-05-09T20:29:07.406 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libgtk2.0-common (2.24.33-2ubuntu2) ... 2024-05-09T20:29:07.657 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libgtk2.0-0:amd64 (2.24.33-2ubuntu2) ... 2024-05-09T20:29:08.042 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up openjdk-8-jre:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-09T20:29:08.142 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/policytool to provide /usr/bin/policytool (policytool) in auto mode 2024-05-09T20:29:08.211 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up openjdk-8-jdk-headless:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-09T20:29:08.303 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/clhsdb to provide /usr/bin/clhsdb (clhsdb) in auto mode 2024-05-09T20:29:08.332 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/extcheck to provide /usr/bin/extcheck (extcheck) in auto mode 2024-05-09T20:29:08.358 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/hsdb to provide /usr/bin/hsdb (hsdb) in auto mode 2024-05-09T20:29:08.382 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/idlj to provide /usr/bin/idlj (idlj) in auto mode 2024-05-09T20:29:08.503 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/javah to provide /usr/bin/javah (javah) in auto mode 2024-05-09T20:29:08.647 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jhat to provide /usr/bin/jhat (jhat) in auto mode 2024-05-09T20:29:08.774 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/jsadebugd to provide /usr/bin/jsadebugd (jsadebugd) in auto mode 2024-05-09T20:29:08.874 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/native2ascii to provide /usr/bin/native2ascii (native2ascii) in auto mode 2024-05-09T20:29:08.917 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/schemagen to provide /usr/bin/schemagen (schemagen) in auto mode 2024-05-09T20:29:08.977 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode 2024-05-09T20:29:09.002 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/wsimport to provide /usr/bin/wsimport (wsimport) in auto mode 2024-05-09T20:29:09.044 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode 2024-05-09T20:29:09.100 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libgail18:amd64 (2.24.33-2ubuntu2) ... 2024-05-09T20:29:09.226 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libgtk2.0-bin (2.24.33-2ubuntu2) ... 2024-05-09T20:29:09.327 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up openjdk-8-jdk:amd64 (8u402-ga-2ubuntu1~22.04) ... 2024-05-09T20:29:09.428 INFO:teuthology.orchestra.run.smithi063.stdout:update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode 2024-05-09T20:29:09.521 INFO:teuthology.orchestra.run.smithi063.stdout:Setting up libgail-common:amd64 (2.24.33-2ubuntu2) ... 2024-05-09T20:29:09.648 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-05-09T20:29:10.741 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-05-09T20:29:10.834 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-05-09T20:29:11.094 INFO:teuthology.orchestra.run.smithi063.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-05-09T20:29:11.581 INFO:teuthology.orchestra.run.smithi063.stdout: Gradle will be installed under /opt/gradle 2024-05-09T20:29:11.619 INFO:teuthology.orchestra.run.smithi063.stderr:--2024-05-09 20:29:11-- https://services.gradle.org/distributions/gradle-6.0.1-bin.zip 2024-05-09T20:29:11.634 INFO:teuthology.orchestra.run.smithi063.stderr:Resolving services.gradle.org (services.gradle.org)... 104.16.73.101, 104.16.72.101, 2606:4700::6810:4965, ... 2024-05-09T20:29:11.644 INFO:teuthology.orchestra.run.smithi063.stderr:Connecting to services.gradle.org (services.gradle.org)|104.16.73.101|:443... connected. 2024-05-09T20:29:11.691 INFO:teuthology.orchestra.run.smithi063.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2024-05-09T20:29:11.691 INFO:teuthology.orchestra.run.smithi063.stderr:Location: https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip [following] 2024-05-09T20:29:11.691 INFO:teuthology.orchestra.run.smithi063.stderr:--2024-05-09 20:29:11-- https://github.com/gradle/gradle-distributions/releases/download/v6.0.1/gradle-6.0.1-bin.zip 2024-05-09T20:29:11.704 INFO:teuthology.orchestra.run.smithi063.stderr:Resolving github.com (github.com)... 140.82.112.4 2024-05-09T20:29:11.715 INFO:teuthology.orchestra.run.smithi063.stderr:Connecting to github.com (github.com)|140.82.112.4|:443... connected. 2024-05-09T20:29:11.757 INFO:teuthology.orchestra.run.smithi063.stderr:HTTP request sent, awaiting response... 302 Found 2024-05-09T20:29:11.758 INFO:teuthology.orchestra.run.smithi063.stderr:Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240509%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240509T202710Z&X-Amz-Expires=300&X-Amz-Signature=e5ef70fb52188a9128fa444341248c0d9b24cd2779b3a2dc73568750622f939a&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream [following] 2024-05-09T20:29:11.758 INFO:teuthology.orchestra.run.smithi063.stderr:--2024-05-09 20:29:11-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/696192900/8f418ba1-c336-448c-95c6-5180382113cb?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAVCODYLSA53PQK4ZA%2F20240509%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240509T202710Z&X-Amz-Expires=300&X-Amz-Signature=e5ef70fb52188a9128fa444341248c0d9b24cd2779b3a2dc73568750622f939a&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=696192900&response-content-disposition=attachment%3B%20filename%3Dgradle-6.0.1-bin.zip&response-content-type=application%2Foctet-stream 2024-05-09T20:29:11.760 INFO:teuthology.orchestra.run.smithi063.stderr:Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.109.133, ... 2024-05-09T20:29:11.779 INFO:teuthology.orchestra.run.smithi063.stderr:Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected. 2024-05-09T20:29:11.877 INFO:teuthology.orchestra.run.smithi063.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-09T20:29:11.877 INFO:teuthology.orchestra.run.smithi063.stderr:Length: 95129672 (91M) [application/octet-stream] 2024-05-09T20:29:11.877 INFO:teuthology.orchestra.run.smithi063.stderr:Saving to: 'gradle-6.0.1-bin.zip' 2024-05-09T20:29:11.877 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-09T20:29:11.895 INFO:teuthology.orchestra.run.smithi063.stderr: 0K .......... .......... .......... .......... .......... 0% 2.68M 34s 2024-05-09T20:29:11.906 INFO:teuthology.orchestra.run.smithi063.stderr: 50K .......... .......... .......... .......... .......... 0% 4.14M 28s 2024-05-09T20:29:11.909 INFO:teuthology.orchestra.run.smithi063.stderr: 100K .......... .......... .......... .......... .......... 0% 16.2M 20s 2024-05-09T20:29:11.913 INFO:teuthology.orchestra.run.smithi063.stderr: 150K .......... .......... .......... .......... .......... 0% 15.9M 17s 2024-05-09T20:29:11.921 INFO:teuthology.orchestra.run.smithi063.stderr: 200K .......... .......... .......... .......... .......... 0% 5.75M 17s 2024-05-09T20:29:11.922 INFO:teuthology.orchestra.run.smithi063.stderr: 250K .......... .......... .......... .......... .......... 0% 41.3M 14s 2024-05-09T20:29:11.925 INFO:teuthology.orchestra.run.smithi063.stderr: 300K .......... .......... .......... .......... .......... 0% 20.7M 13s 2024-05-09T20:29:11.926 INFO:teuthology.orchestra.run.smithi063.stderr: 350K .......... .......... .......... .......... .......... 0% 27.7M 12s 2024-05-09T20:29:11.929 INFO:teuthology.orchestra.run.smithi063.stderr: 400K .......... .......... .......... .......... .......... 0% 24.6M 11s 2024-05-09T20:29:11.929 INFO:teuthology.orchestra.run.smithi063.stderr: 450K .......... .......... .......... .......... .......... 0% 58.8M 10s 2024-05-09T20:29:11.936 INFO:teuthology.orchestra.run.smithi063.stderr: 500K .......... .......... .......... .......... .......... 0% 7.49M 10s 2024-05-09T20:29:11.938 INFO:teuthology.orchestra.run.smithi063.stderr: 550K .......... .......... .......... .......... .......... 0% 46.3M 9s 2024-05-09T20:29:11.938 INFO:teuthology.orchestra.run.smithi063.stderr: 600K .......... .......... .......... .......... .......... 0% 49.3M 9s 2024-05-09T20:29:11.940 INFO:teuthology.orchestra.run.smithi063.stderr: 650K .......... .......... .......... .......... .......... 0% 31.8M 8s 2024-05-09T20:29:11.940 INFO:teuthology.orchestra.run.smithi063.stderr: 700K .......... .......... .......... .......... .......... 0% 74.2M 8s 2024-05-09T20:29:11.941 INFO:teuthology.orchestra.run.smithi063.stderr: 750K .......... .......... .......... .......... .......... 0% 37.6M 7s 2024-05-09T20:29:11.945 INFO:teuthology.orchestra.run.smithi063.stderr: 800K .......... .......... .......... .......... .......... 0% 35.7M 7s 2024-05-09T20:29:11.945 INFO:teuthology.orchestra.run.smithi063.stderr: 850K .......... .......... .......... .......... .......... 0% 72.8M 7s 2024-05-09T20:29:11.945 INFO:teuthology.orchestra.run.smithi063.stderr: 900K .......... .......... .......... .......... .......... 1% 83.2M 7s 2024-05-09T20:29:11.945 INFO:teuthology.orchestra.run.smithi063.stderr: 950K .......... .......... .......... .......... .......... 1% 76.5M 6s 2024-05-09T20:29:11.951 INFO:teuthology.orchestra.run.smithi063.stderr: 1000K .......... .......... .......... .......... .......... 1% 9.41M 6s 2024-05-09T20:29:11.951 INFO:teuthology.orchestra.run.smithi063.stderr: 1050K .......... .......... .......... .......... .......... 1% 61.9M 6s 2024-05-09T20:29:11.955 INFO:teuthology.orchestra.run.smithi063.stderr: 1100K .......... .......... .......... .......... .......... 1% 64.7M 6s 2024-05-09T20:29:11.955 INFO:teuthology.orchestra.run.smithi063.stderr: 1150K .......... .......... .......... .......... .......... 1% 80.9M 6s 2024-05-09T20:29:11.955 INFO:teuthology.orchestra.run.smithi063.stderr: 1200K .......... .......... .......... .......... .......... 1% 62.8M 6s 2024-05-09T20:29:11.956 INFO:teuthology.orchestra.run.smithi063.stderr: 1250K .......... .......... .......... .......... .......... 1% 56.9M 5s 2024-05-09T20:29:11.956 INFO:teuthology.orchestra.run.smithi063.stderr: 1300K .......... .......... .......... .......... .......... 1% 71.4M 5s 2024-05-09T20:29:11.959 INFO:teuthology.orchestra.run.smithi063.stderr: 1350K .......... .......... .......... .......... .......... 1% 82.5M 5s 2024-05-09T20:29:11.959 INFO:teuthology.orchestra.run.smithi063.stderr: 1400K .......... .......... .......... .......... .......... 1% 79.8M 5s 2024-05-09T20:29:11.960 INFO:teuthology.orchestra.run.smithi063.stderr: 1450K .......... .......... .......... .......... .......... 1% 68.3M 5s 2024-05-09T20:29:11.960 INFO:teuthology.orchestra.run.smithi063.stderr: 1500K .......... .......... .......... .......... .......... 1% 52.8M 5s 2024-05-09T20:29:11.960 INFO:teuthology.orchestra.run.smithi063.stderr: 1550K .......... .......... .......... .......... .......... 1% 45.8M 5s 2024-05-09T20:29:11.964 INFO:teuthology.orchestra.run.smithi063.stderr: 1600K .......... .......... .......... .......... .......... 1% 72.2M 5s 2024-05-09T20:29:11.964 INFO:teuthology.orchestra.run.smithi063.stderr: 1650K .......... .......... .......... .......... .......... 1% 77.9M 4s 2024-05-09T20:29:11.964 INFO:teuthology.orchestra.run.smithi063.stderr: 1700K .......... .......... .......... .......... .......... 1% 60.1M 4s 2024-05-09T20:29:11.964 INFO:teuthology.orchestra.run.smithi063.stderr: 1750K .......... .......... .......... .......... .......... 1% 77.2M 4s 2024-05-09T20:29:11.964 INFO:teuthology.orchestra.run.smithi063.stderr: 1800K .......... .......... .......... .......... .......... 1% 77.4M 4s 2024-05-09T20:29:11.969 INFO:teuthology.orchestra.run.smithi063.stderr: 1850K .......... .......... .......... .......... .......... 2% 80.3M 4s 2024-05-09T20:29:11.969 INFO:teuthology.orchestra.run.smithi063.stderr: 1900K .......... .......... .......... .......... .......... 2% 76.2M 4s 2024-05-09T20:29:11.969 INFO:teuthology.orchestra.run.smithi063.stderr: 1950K .......... .......... .......... .......... .......... 2% 85.9M 4s 2024-05-09T20:29:11.969 INFO:teuthology.orchestra.run.smithi063.stderr: 2000K .......... .......... .......... .......... .......... 2% 81.3M 4s 2024-05-09T20:29:11.969 INFO:teuthology.orchestra.run.smithi063.stderr: 2050K .......... .......... .......... .......... .......... 2% 71.9M 4s 2024-05-09T20:29:11.969 INFO:teuthology.orchestra.run.smithi063.stderr: 2100K .......... .......... .......... .......... .......... 2% 63.3M 4s 2024-05-09T20:29:11.976 INFO:teuthology.orchestra.run.smithi063.stderr: 2150K .......... .......... .......... .......... .......... 2% 70.7M 4s 2024-05-09T20:29:11.976 INFO:teuthology.orchestra.run.smithi063.stderr: 2200K .......... .......... .......... .......... .......... 2% 76.6M 4s 2024-05-09T20:29:11.976 INFO:teuthology.orchestra.run.smithi063.stderr: 2250K .......... .......... .......... .......... .......... 2% 76.4M 4s 2024-05-09T20:29:11.976 INFO:teuthology.orchestra.run.smithi063.stderr: 2300K .......... .......... .......... .......... .......... 2% 68.5M 4s 2024-05-09T20:29:11.976 INFO:teuthology.orchestra.run.smithi063.stderr: 2350K .......... .......... .......... .......... .......... 2% 76.8M 3s 2024-05-09T20:29:11.977 INFO:teuthology.orchestra.run.smithi063.stderr: 2400K .......... .......... .......... .......... .......... 2% 73.5M 3s 2024-05-09T20:29:11.977 INFO:teuthology.orchestra.run.smithi063.stderr: 2450K .......... .......... .......... .......... .......... 2% 59.6M 3s 2024-05-09T20:29:11.977 INFO:teuthology.orchestra.run.smithi063.stderr: 2500K .......... .......... .......... .......... .......... 2% 71.9M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2550K .......... .......... .......... .......... .......... 2% 77.3M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2600K .......... .......... .......... .......... .......... 2% 79.1M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2650K .......... .......... .......... .......... .......... 2% 69.5M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2700K .......... .......... .......... .......... .......... 2% 75.7M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2750K .......... .......... .......... .......... .......... 3% 75.5M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2800K .......... .......... .......... .......... .......... 3% 75.5M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2850K .......... .......... .......... .......... .......... 3% 65.2M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2900K .......... .......... .......... .......... .......... 3% 78.1M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 2950K .......... .......... .......... .......... .......... 3% 78.7M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 3000K .......... .......... .......... .......... .......... 3% 68.3M 3s 2024-05-09T20:29:11.985 INFO:teuthology.orchestra.run.smithi063.stderr: 3050K .......... .......... .......... .......... .......... 3% 66.0M 3s 2024-05-09T20:29:13.237 INFO:teuthology.orchestra.run.smithi063.stderr: 3100K .......... .......... .......... .......... .......... 3% 74.6M 3s 2024-05-09T20:29:13.237 INFO:teuthology.orchestra.run.smithi063.stderr: 3150K .......... .......... .......... .......... .......... 3% 77.1M 3s 2024-05-09T20:29:13.237 INFO:teuthology.orchestra.run.smithi063.stderr: 3200K .......... .......... .......... .......... .......... 3% 80.6M 3s 2024-05-09T20:29:13.237 INFO:teuthology.orchestra.run.smithi063.stderr: 3250K .......... .......... .......... .......... .......... 3% 63.3M 3s 2024-05-09T20:29:13.237 INFO:teuthology.orchestra.run.smithi063.stderr: 3300K .......... .......... .......... .......... .......... 3% 73.7M 3s 2024-05-09T20:29:13.237 INFO:teuthology.orchestra.run.smithi063.stderr: 3350K .......... .......... .......... .......... .......... 3% 77.7M 3s 2024-05-09T20:29:13.237 INFO:teuthology.orchestra.run.smithi063.stderr: 3400K .......... .......... .......... .......... .......... 3% 78.7M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3450K .......... .......... .......... .......... .......... 3% 72.1M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3500K .......... .......... .......... .......... .......... 3% 79.7M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3550K .......... .......... .......... .......... .......... 3% 75.4M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3600K .......... .......... .......... .......... .......... 3% 72.1M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3650K .......... .......... .......... .......... .......... 3% 64.8M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3700K .......... .......... .......... .......... .......... 4% 79.5M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3750K .......... .......... .......... .......... .......... 4% 76.7M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3800K .......... .......... .......... .......... .......... 4% 77.9M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3850K .......... .......... .......... .......... .......... 4% 70.4M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3900K .......... .......... .......... .......... .......... 4% 74.3M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 3950K .......... .......... .......... .......... .......... 4% 85.7M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 4000K .......... .......... .......... .......... .......... 4% 81.9M 3s 2024-05-09T20:29:13.238 INFO:teuthology.orchestra.run.smithi063.stderr: 4050K .......... .......... .......... .......... .......... 4% 70.4M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4100K .......... .......... .......... .......... .......... 4% 84.1M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4150K .......... .......... .......... .......... .......... 4% 86.9M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4200K .......... .......... .......... .......... .......... 4% 63.5M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4250K .......... .......... .......... .......... .......... 4% 83.7M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4300K .......... .......... .......... .......... .......... 4% 86.3M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4350K .......... .......... .......... .......... .......... 4% 70.5M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4400K .......... .......... .......... .......... .......... 4% 89.6M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4450K .......... .......... .......... .......... .......... 4% 88.5M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4500K .......... .......... .......... .......... .......... 4% 87.2M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4550K .......... .......... .......... .......... .......... 4% 75.3M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4600K .......... .......... .......... .......... .......... 5% 90.7M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4650K .......... .......... .......... .......... .......... 5% 90.8M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4700K .......... .......... .......... .......... .......... 5% 95.0M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4750K .......... .......... .......... .......... .......... 5% 78.1M 2s 2024-05-09T20:29:13.239 INFO:teuthology.orchestra.run.smithi063.stderr: 4800K .......... .......... .......... .......... .......... 5% 87.8M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 4850K .......... .......... .......... .......... .......... 5% 92.2M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 4900K .......... .......... .......... .......... .......... 5% 87.5M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 4950K .......... .......... .......... .......... .......... 5% 86.0M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5000K .......... .......... .......... .......... .......... 5% 97.7M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5050K .......... .......... .......... .......... .......... 5% 93.4M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5100K .......... .......... .......... .......... .......... 5% 78.2M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5150K .......... .......... .......... .......... .......... 5% 96.7M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5200K .......... .......... .......... .......... .......... 5% 101M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5250K .......... .......... .......... .......... .......... 5% 99.7M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5300K .......... .......... .......... .......... .......... 5% 85.7M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5350K .......... .......... .......... .......... .......... 5% 106M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5400K .......... .......... .......... .......... .......... 5% 103M 2s 2024-05-09T20:29:13.240 INFO:teuthology.orchestra.run.smithi063.stderr: 5450K .......... .......... .......... .......... .......... 5% 102M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5500K .......... .......... .......... .......... .......... 5% 82.6M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5550K .......... .......... .......... .......... .......... 6% 98.6M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5600K .......... .......... .......... .......... .......... 6% 99.5M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5650K .......... .......... .......... .......... .......... 6% 103M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5700K .......... .......... .......... .......... .......... 6% 89.1M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5750K .......... .......... .......... .......... .......... 6% 107M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5800K .......... .......... .......... .......... .......... 6% 102M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5850K .......... .......... .......... .......... .......... 6% 107M 2s 2024-05-09T20:29:13.241 INFO:teuthology.orchestra.run.smithi063.stderr: 5900K .......... .......... .......... .......... .......... 6% 87.2M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 5950K .......... .......... .......... .......... .......... 6% 109M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6000K .......... .......... .......... .......... .......... 6% 104M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6050K .......... .......... .......... .......... .......... 6% 106M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6100K .......... .......... .......... .......... .......... 6% 110M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6150K .......... .......... .......... .......... .......... 6% 169M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6200K .......... .......... .......... .......... .......... 6% 196M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6250K .......... .......... .......... .......... .......... 6% 165M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6300K .......... .......... .......... .......... .......... 6% 178M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6350K .......... .......... .......... .......... .......... 6% 182M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6400K .......... .......... .......... .......... .......... 6% 191M 2s 2024-05-09T20:29:13.242 INFO:teuthology.orchestra.run.smithi063.stderr: 6450K .......... .......... .......... .......... .......... 6% 181M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6500K .......... .......... .......... .......... .......... 7% 190M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6550K .......... .......... .......... .......... .......... 7% 190M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6600K .......... .......... .......... .......... .......... 7% 196M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6650K .......... .......... .......... .......... .......... 7% 165M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6700K .......... .......... .......... .......... .......... 7% 190M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6750K .......... .......... .......... .......... .......... 7% 197M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6800K .......... .......... .......... .......... .......... 7% 198M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6850K .......... .......... .......... .......... .......... 7% 165M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6900K .......... .......... .......... .......... .......... 7% 190M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 6950K .......... .......... .......... .......... .......... 7% 206M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 7000K .......... .......... .......... .......... .......... 7% 191M 2s 2024-05-09T20:29:13.243 INFO:teuthology.orchestra.run.smithi063.stderr: 7050K .......... .......... .......... .......... .......... 7% 175M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7100K .......... .......... .......... .......... .......... 7% 214M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7150K .......... .......... .......... .......... .......... 7% 185M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7200K .......... .......... .......... .......... .......... 7% 197M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7250K .......... .......... .......... .......... .......... 7% 172M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7300K .......... .......... .......... .......... .......... 7% 201M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7350K .......... .......... .......... .......... .......... 7% 189M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7400K .......... .......... .......... .......... .......... 8% 207M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7450K .......... .......... .......... .......... .......... 8% 152M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7500K .......... .......... .......... .......... .......... 8% 204M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7550K .......... .......... .......... .......... .......... 8% 210M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7600K .......... .......... .......... .......... .......... 8% 177M 2s 2024-05-09T20:29:13.244 INFO:teuthology.orchestra.run.smithi063.stderr: 7650K .......... .......... .......... .......... .......... 8% 180M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 7700K .......... .......... .......... .......... .......... 8% 210M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 7750K .......... .......... .......... .......... .......... 8% 217M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 7800K .......... .......... .......... .......... .......... 8% 233M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 7850K .......... .......... .......... .......... .......... 8% 172M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 7900K .......... .......... .......... .......... .......... 8% 211M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 7950K .......... .......... .......... .......... .......... 8% 179M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 8000K .......... .......... .......... .......... .......... 8% 211M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 8050K .......... .......... .......... .......... .......... 8% 200M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 8100K .......... .......... .......... .......... .......... 8% 204M 2s 2024-05-09T20:29:13.245 INFO:teuthology.orchestra.run.smithi063.stderr: 8150K .......... .......... .......... .......... .......... 8% 223M 2s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8200K .......... .......... .......... .......... .......... 8% 215M 2s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8250K .......... .......... .......... .......... .......... 8% 180M 2s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8300K .......... .......... .......... .......... .......... 8% 169M 1s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8350K .......... .......... .......... .......... .......... 9% 205M 1s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8400K .......... .......... .......... .......... .......... 9% 179M 1s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8450K .......... .......... .......... .......... .......... 9% 211M 1s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8500K .......... .......... .......... .......... .......... 9% 232M 1s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8550K .......... .......... .......... .......... .......... 9% 226M 1s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8600K .......... .......... .......... .......... .......... 9% 225M 1s 2024-05-09T20:29:13.246 INFO:teuthology.orchestra.run.smithi063.stderr: 8650K .......... .......... .......... .......... .......... 9% 193M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 8700K .......... .......... .......... .......... .......... 9% 227M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 8750K .......... .......... .......... .......... .......... 9% 237M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 8800K .......... .......... .......... .......... .......... 9% 268M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 8850K .......... .......... .......... .......... .......... 9% 233M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 8900K .......... .......... .......... .......... .......... 9% 74.0M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 8950K .......... .......... .......... .......... .......... 9% 122M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 9000K .......... .......... .......... .......... .......... 9% 111M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 9050K .......... .......... .......... .......... .......... 9% 78.7M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 9100K .......... .......... .......... .......... .......... 9% 96.1M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 9150K .......... .......... .......... .......... .......... 9% 144M 1s 2024-05-09T20:29:13.247 INFO:teuthology.orchestra.run.smithi063.stderr: 9200K .......... .......... .......... .......... .......... 9% 110M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9250K .......... .......... .......... .......... .......... 10% 134M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9300K .......... .......... .......... .......... .......... 10% 113M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9350K .......... .......... .......... .......... .......... 10% 104M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9400K .......... .......... .......... .......... .......... 10% 87.2M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9450K .......... .......... .......... .......... .......... 10% 111M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9500K .......... .......... .......... .......... .......... 10% 121M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9550K .......... .......... .......... .......... .......... 10% 132M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9600K .......... .......... .......... .......... .......... 10% 106M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9650K .......... .......... .......... .......... .......... 10% 113M 1s 2024-05-09T20:29:13.248 INFO:teuthology.orchestra.run.smithi063.stderr: 9700K .......... .......... .......... .......... .......... 10% 117M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 9750K .......... .......... .......... .......... .......... 10% 113M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 9800K .......... .......... .......... .......... .......... 10% 96.6M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 9850K .......... .......... .......... .......... .......... 10% 113M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 9900K .......... .......... .......... .......... .......... 10% 112M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 9950K .......... .......... .......... .......... .......... 10% 116M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 10000K .......... .......... .......... .......... .......... 10% 105M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 10050K .......... .......... .......... .......... .......... 10% 117M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 10100K .......... .......... .......... .......... .......... 10% 107M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 10150K .......... .......... .......... .......... .......... 10% 122M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 10200K .......... .......... .......... .......... .......... 11% 90.1M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 10250K .......... .......... .......... .......... .......... 11% 123M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 10300K .......... .......... .......... .......... .......... 11% 84.7M 1s 2024-05-09T20:29:13.249 INFO:teuthology.orchestra.run.smithi063.stderr: 10350K .......... .......... .......... .......... .......... 11% 118M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10400K .......... .......... .......... .......... .......... 11% 98.3M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10450K .......... .......... .......... .......... .......... 11% 16.9M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10500K .......... .......... .......... .......... .......... 11% 55.4M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10550K .......... .......... .......... .......... .......... 11% 116M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10600K .......... .......... .......... .......... .......... 11% 150M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10650K .......... .......... .......... .......... .......... 11% 177M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10700K .......... .......... .......... .......... .......... 11% 177M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10750K .......... .......... .......... .......... .......... 11% 179M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10800K .......... .......... .......... .......... .......... 11% 159M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10850K .......... .......... .......... .......... .......... 11% 171M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10900K .......... .......... .......... .......... .......... 11% 175M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 10950K .......... .......... .......... .......... .......... 11% 174M 1s 2024-05-09T20:29:13.250 INFO:teuthology.orchestra.run.smithi063.stderr: 11000K .......... .......... .......... .......... .......... 11% 146M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11050K .......... .......... .......... .......... .......... 11% 176M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11100K .......... .......... .......... .......... .......... 12% 175M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11150K .......... .......... .......... .......... .......... 12% 179M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11200K .......... .......... .......... .......... .......... 12% 146M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11250K .......... .......... .......... .......... .......... 12% 171M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11300K .......... .......... .......... .......... .......... 12% 174M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11350K .......... .......... .......... .......... .......... 12% 179M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11400K .......... .......... .......... .......... .......... 12% 151M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11450K .......... .......... .......... .......... .......... 12% 192M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11500K .......... .......... .......... .......... .......... 12% 140M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11550K .......... .......... .......... .......... .......... 12% 99.3M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11600K .......... .......... .......... .......... .......... 12% 126M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11650K .......... .......... .......... .......... .......... 12% 116M 1s 2024-05-09T20:29:13.251 INFO:teuthology.orchestra.run.smithi063.stderr: 11700K .......... .......... .......... .......... .......... 12% 112M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 11750K .......... .......... .......... .......... .......... 12% 101M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 11800K .......... .......... .......... .......... .......... 12% 104M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 11850K .......... .......... .......... .......... .......... 12% 119M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 11900K .......... .......... .......... .......... .......... 12% 114M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 11950K .......... .......... .......... .......... .......... 12% 109M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 12000K .......... .......... .......... .......... .......... 12% 91.2M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 12050K .......... .......... .......... .......... .......... 13% 121M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 12100K .......... .......... .......... .......... .......... 13% 92.2M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 12150K .......... .......... .......... .......... .......... 13% 130M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 12200K .......... .......... .......... .......... .......... 13% 124M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 12250K .......... .......... .......... .......... .......... 13% 104M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 12300K .......... .......... .......... .......... .......... 13% 43.7M 1s 2024-05-09T20:29:13.252 INFO:teuthology.orchestra.run.smithi063.stderr: 12350K .......... .......... .......... .......... .......... 13% 137M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12400K .......... .......... .......... .......... .......... 13% 120M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12450K .......... .......... .......... .......... .......... 13% 108M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12500K .......... .......... .......... .......... .......... 13% 91.4M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12550K .......... .......... .......... .......... .......... 13% 114M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12600K .......... .......... .......... .......... .......... 13% 118M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12650K .......... .......... .......... .......... .......... 13% 112M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12700K .......... .......... .......... .......... .......... 13% 93.1M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12750K .......... .......... .......... .......... .......... 13% 87.1M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12800K .......... .......... .......... .......... .......... 13% 109M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12850K .......... .......... .......... .......... .......... 13% 119M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12900K .......... .......... .......... .......... .......... 13% 96.9M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 12950K .......... .......... .......... .......... .......... 13% 110M 1s 2024-05-09T20:29:13.253 INFO:teuthology.orchestra.run.smithi063.stderr: 13000K .......... .......... .......... .......... .......... 14% 114M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13050K .......... .......... .......... .......... .......... 14% 119M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13100K .......... .......... .......... .......... .......... 14% 109M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13150K .......... .......... .......... .......... .......... 14% 46.2M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13200K .......... .......... .......... .......... .......... 14% 118M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13250K .......... .......... .......... .......... .......... 14% 109M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13300K .......... .......... .......... .......... .......... 14% 95.4M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13350K .......... .......... .......... .......... .......... 14% 117M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13400K .......... .......... .......... .......... .......... 14% 109M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13450K .......... .......... .......... .......... .......... 14% 102M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13500K .......... .......... .......... .......... .......... 14% 104M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13550K .......... .......... .......... .......... .......... 14% 119M 1s 2024-05-09T20:29:13.254 INFO:teuthology.orchestra.run.smithi063.stderr: 13600K .......... .......... .......... .......... .......... 14% 115M 1s 2024-05-09T20:29:13.255 INFO:teuthology.orchestra.run.smithi063.stderr: 13650K .......... .......... .......... .......... .......... 14% 103M 1s 2024-05-09T20:29:13.255 INFO:teuthology.orchestra.run.smithi063.stderr: 13700K .......... .......... .......... .......... .......... 14% 101M 1s 2024-05-09T20:29:13.255 INFO:teuthology.orchestra.run.smithi063.stderr: 13750K .......... .......... .......... .......... .......... 14% 118M 1s 2024-05-09T20:29:13.255 INFO:teuthology.orchestra.run.smithi063.stderr: 13800K .......... .......... .......... .......... .......... 14% 133M 1s 2024-05-09T20:29:13.255 INFO:teuthology.orchestra.run.smithi063.stderr: 13850K .......... .......... .......... .......... .......... 14% 90.0M 1s 2024-05-09T20:29:13.255 INFO:teuthology.orchestra.run.smithi063.stderr: 13900K .......... .......... .......... .......... .......... 15% 116M 1s 2024-05-09T20:29:13.255 INFO:teuthology.orchestra.run.smithi063.stderr: 13950K .......... .......... .......... .......... .......... 15% 105M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14000K .......... .......... .......... .......... .......... 15% 124M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14050K .......... .......... .......... .......... .......... 15% 103M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14100K .......... .......... .......... .......... .......... 15% 124M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14150K .......... .......... .......... .......... .......... 15% 111M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14200K .......... .......... .......... .......... .......... 15% 122M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14250K .......... .......... .......... .......... .......... 15% 89.1M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14300K .......... .......... .......... .......... .......... 15% 112M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14350K .......... .......... .......... .......... .......... 15% 116M 1s 2024-05-09T20:29:13.256 INFO:teuthology.orchestra.run.smithi063.stderr: 14400K .......... .......... .......... .......... .......... 15% 101M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14450K .......... .......... .......... .......... .......... 15% 115M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14500K .......... .......... .......... .......... .......... 15% 128M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14550K .......... .......... .......... .......... .......... 15% 111M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14600K .......... .......... .......... .......... .......... 15% 118M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14650K .......... .......... .......... .......... .......... 15% 90.5M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14700K .......... .......... .......... .......... .......... 15% 117M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14750K .......... .......... .......... .......... .......... 15% 112M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14800K .......... .......... .......... .......... .......... 15% 112M 1s 2024-05-09T20:29:13.257 INFO:teuthology.orchestra.run.smithi063.stderr: 14850K .......... .......... .......... .......... .......... 16% 107M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 14900K .......... .......... .......... .......... .......... 16% 119M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 14950K .......... .......... .......... .......... .......... 16% 111M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15000K .......... .......... .......... .......... .......... 16% 110M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15050K .......... .......... .......... .......... .......... 16% 90.0M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15100K .......... .......... .......... .......... .......... 16% 115M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15150K .......... .......... .......... .......... .......... 16% 129M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15200K .......... .......... .......... .......... .......... 16% 91.0M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15250K .......... .......... .......... .......... .......... 16% 109M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15300K .......... .......... .......... .......... .......... 16% 115M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15350K .......... .......... .......... .......... .......... 16% 72.4M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15400K .......... .......... .......... .......... .......... 16% 141M 1s 2024-05-09T20:29:13.258 INFO:teuthology.orchestra.run.smithi063.stderr: 15450K .......... .......... .......... .......... .......... 16% 121M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15500K .......... .......... .......... .......... .......... 16% 91.1M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15550K .......... .......... .......... .......... .......... 16% 114M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15600K .......... .......... .......... .......... .......... 16% 110M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15650K .......... .......... .......... .......... .......... 16% 119M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15700K .......... .......... .......... .......... .......... 16% 107M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15750K .......... .......... .......... .......... .......... 17% 109M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15800K .......... .......... .......... .......... .......... 17% 118M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15850K .......... .......... .......... .......... .......... 17% 116M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15900K .......... .......... .......... .......... .......... 17% 94.6M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 15950K .......... .......... .......... .......... .......... 17% 113M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 16000K .......... .......... .......... .......... .......... 17% 115M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 16050K .......... .......... .......... .......... .......... 17% 110M 1s 2024-05-09T20:29:13.259 INFO:teuthology.orchestra.run.smithi063.stderr: 16100K .......... .......... .......... .......... .......... 17% 107M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16150K .......... .......... .......... .......... .......... 17% 112M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16200K .......... .......... .......... .......... .......... 17% 123M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16250K .......... .......... .......... .......... .......... 17% 111M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16300K .......... .......... .......... .......... .......... 17% 83.1M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16350K .......... .......... .......... .......... .......... 17% 115M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16400K .......... .......... .......... .......... .......... 17% 119M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16450K .......... .......... .......... .......... .......... 17% 109M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16500K .......... .......... .......... .......... .......... 17% 128M 1s 2024-05-09T20:29:13.260 INFO:teuthology.orchestra.run.smithi063.stderr: 16550K .......... .......... .......... .......... .......... 17% 117M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 16600K .......... .......... .......... .......... .......... 17% 102M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 16650K .......... .......... .......... .......... .......... 17% 114M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 16700K .......... .......... .......... .......... .......... 18% 115M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 16750K .......... .......... .......... .......... .......... 18% 104M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 16800K .......... .......... .......... .......... .......... 18% 115M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 16850K .......... .......... .......... .......... .......... 18% 120M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 16900K .......... .......... .......... .......... .......... 18% 115M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 16950K .......... .......... .......... .......... .......... 18% 91.7M 1s 2024-05-09T20:29:13.261 INFO:teuthology.orchestra.run.smithi063.stderr: 17000K .......... .......... .......... .......... .......... 18% 118M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17050K .......... .......... .......... .......... .......... 18% 110M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17100K .......... .......... .......... .......... .......... 18% 104M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17150K .......... .......... .......... .......... .......... 18% 99.9M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17200K .......... .......... .......... .......... .......... 18% 116M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17250K .......... .......... .......... .......... .......... 18% 108M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17300K .......... .......... .......... .......... .......... 18% 122M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17350K .......... .......... .......... .......... .......... 18% 113M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17400K .......... .......... .......... .......... .......... 18% 89.7M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17450K .......... .......... .......... .......... .......... 18% 145M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17500K .......... .......... .......... .......... .......... 18% 118M 1s 2024-05-09T20:29:13.262 INFO:teuthology.orchestra.run.smithi063.stderr: 17550K .......... .......... .......... .......... .......... 18% 113M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 17600K .......... .......... .......... .......... .......... 18% 89.6M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 17650K .......... .......... .......... .......... .......... 19% 101M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 17700K .......... .......... .......... .......... .......... 19% 125M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 17750K .......... .......... .......... .......... .......... 19% 121M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 17800K .......... .......... .......... .......... .......... 19% 107M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 17850K .......... .......... .......... .......... .......... 19% 117M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 17900K .......... .......... .......... .......... .......... 19% 111M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 17950K .......... .......... .......... .......... .......... 19% 112M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 18000K .......... .......... .......... .......... .......... 19% 89.3M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 18050K .......... .......... .......... .......... .......... 19% 115M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 18100K .......... .......... .......... .......... .......... 19% 113M 1s 2024-05-09T20:29:13.263 INFO:teuthology.orchestra.run.smithi063.stderr: 18150K .......... .......... .......... .......... .......... 19% 111M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18200K .......... .......... .......... .......... .......... 19% 115M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18250K .......... .......... .......... .......... .......... 19% 110M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18300K .......... .......... .......... .......... .......... 19% 114M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18350K .......... .......... .......... .......... .......... 19% 107M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18400K .......... .......... .......... .......... .......... 19% 110M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18450K .......... .......... .......... .......... .......... 19% 110M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18500K .......... .......... .......... .......... .......... 19% 109M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18550K .......... .......... .......... .......... .......... 20% 97.9M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18600K .......... .......... .......... .......... .......... 20% 109M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18650K .......... .......... .......... .......... .......... 20% 121M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18700K .......... .......... .......... .......... .......... 20% 107M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18750K .......... .......... .......... .......... .......... 20% 92.3M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18800K .......... .......... .......... .......... .......... 20% 146M 1s 2024-05-09T20:29:13.264 INFO:teuthology.orchestra.run.smithi063.stderr: 18850K .......... .......... .......... .......... .......... 20% 113M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 18900K .......... .......... .......... .......... .......... 20% 101M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 18950K .......... .......... .......... .......... .......... 20% 107M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19000K .......... .......... .......... .......... .......... 20% 122M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19050K .......... .......... .......... .......... .......... 20% 92.8M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19100K .......... .......... .......... .......... .......... 20% 127M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19150K .......... .......... .......... .......... .......... 20% 46.5M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19200K .......... .......... .......... .......... .......... 20% 111M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19250K .......... .......... .......... .......... .......... 20% 97.1M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19300K .......... .......... .......... .......... .......... 20% 79.1M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19350K .......... .......... .......... .......... .......... 20% 99.7M 1s 2024-05-09T20:29:13.265 INFO:teuthology.orchestra.run.smithi063.stderr: 19400K .......... .......... .......... .......... .......... 20% 111M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19450K .......... .......... .......... .......... .......... 20% 110M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19500K .......... .......... .......... .......... .......... 21% 121M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19550K .......... .......... .......... .......... .......... 21% 109M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19600K .......... .......... .......... .......... .......... 21% 113M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19650K .......... .......... .......... .......... .......... 21% 80.6M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19700K .......... .......... .......... .......... .......... 21% 126M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19750K .......... .......... .......... .......... .......... 21% 129M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19800K .......... .......... .......... .......... .......... 21% 115M 1s 2024-05-09T20:29:13.266 INFO:teuthology.orchestra.run.smithi063.stderr: 19850K .......... .......... .......... .......... .......... 21% 112M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 19900K .......... .......... .......... .......... .......... 21% 124M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 19950K .......... .......... .......... .......... .......... 21% 88.0M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20000K .......... .......... .......... .......... .......... 21% 135M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20050K .......... .......... .......... .......... .......... 21% 72.0M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20100K .......... .......... .......... .......... .......... 21% 110M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20150K .......... .......... .......... .......... .......... 21% 127M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20200K .......... .......... .......... .......... .......... 21% 73.5M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20250K .......... .......... .......... .......... .......... 21% 173M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20300K .......... .......... .......... .......... .......... 21% 123M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20350K .......... .......... .......... .......... .......... 21% 70.1M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20400K .......... .......... .......... .......... .......... 22% 124M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20450K .......... .......... .......... .......... .......... 22% 83.7M 1s 2024-05-09T20:29:13.267 INFO:teuthology.orchestra.run.smithi063.stderr: 20500K .......... .......... .......... .......... .......... 22% 113M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20550K .......... .......... .......... .......... .......... 22% 110M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20600K .......... .......... .......... .......... .......... 22% 141M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20650K .......... .......... .......... .......... .......... 22% 74.3M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20700K .......... .......... .......... .......... .......... 22% 90.8M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20750K .......... .......... .......... .......... .......... 22% 111M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20800K .......... .......... .......... .......... .......... 22% 117M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20850K .......... .......... .......... .......... .......... 22% 103M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20900K .......... .......... .......... .......... .......... 22% 108M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 20950K .......... .......... .......... .......... .......... 22% 76.5M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 21000K .......... .......... .......... .......... .......... 22% 98.8M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 21050K .......... .......... .......... .......... .......... 22% 115M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 21100K .......... .......... .......... .......... .......... 22% 101M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 21150K .......... .......... .......... .......... .......... 22% 99.0M 1s 2024-05-09T20:29:13.268 INFO:teuthology.orchestra.run.smithi063.stderr: 21200K .......... .......... .......... .......... .......... 22% 87.2M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21250K .......... .......... .......... .......... .......... 22% 101M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21300K .......... .......... .......... .......... .......... 22% 107M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21350K .......... .......... .......... .......... .......... 23% 101M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21400K .......... .......... .......... .......... .......... 23% 112M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21450K .......... .......... .......... .......... .......... 23% 114M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21500K .......... .......... .......... .......... .......... 23% 111M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21550K .......... .......... .......... .......... .......... 23% 122M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21600K .......... .......... .......... .......... .......... 23% 103M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21650K .......... .......... .......... .......... .......... 23% 104M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21700K .......... .......... .......... .......... .......... 23% 119M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21750K .......... .......... .......... .......... .......... 23% 108M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21800K .......... .......... .......... .......... .......... 23% 120M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21850K .......... .......... .......... .......... .......... 23% 84.2M 1s 2024-05-09T20:29:13.269 INFO:teuthology.orchestra.run.smithi063.stderr: 21900K .......... .......... .......... .......... .......... 23% 116M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 21950K .......... .......... .......... .......... .......... 23% 107M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22000K .......... .......... .......... .......... .......... 23% 106M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22050K .......... .......... .......... .......... .......... 23% 112M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22100K .......... .......... .......... .......... .......... 23% 112M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22150K .......... .......... .......... .......... .......... 23% 103M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22200K .......... .......... .......... .......... .......... 23% 112M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22250K .......... .......... .......... .......... .......... 24% 88.8M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22300K .......... .......... .......... .......... .......... 24% 126M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22350K .......... .......... .......... .......... .......... 24% 107M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22400K .......... .......... .......... .......... .......... 24% 125M 1s 2024-05-09T20:29:13.270 INFO:teuthology.orchestra.run.smithi063.stderr: 22450K .......... .......... .......... .......... .......... 24% 58.2M 1s 2024-05-09T20:29:13.271 INFO:teuthology.orchestra.run.smithi063.stderr: 22500K .......... .......... .......... .......... .......... 24% 121M 1s 2024-05-09T20:29:13.271 INFO:teuthology.orchestra.run.smithi063.stderr: 22550K .......... .......... .......... .......... .......... 24% 42.2M 1s 2024-05-09T20:29:13.271 INFO:teuthology.orchestra.run.smithi063.stderr: 22600K .......... .......... .......... .......... .......... 24% 109M 1s 2024-05-09T20:29:13.271 INFO:teuthology.orchestra.run.smithi063.stderr: 22650K .......... .......... .......... .......... .......... 24% 124M 1s 2024-05-09T20:29:13.271 INFO:teuthology.orchestra.run.smithi063.stderr: 22700K .......... .......... .......... .......... .......... 24% 105M 1s 2024-05-09T20:29:13.271 INFO:teuthology.orchestra.run.smithi063.stderr: 22750K .......... .......... .......... .......... .......... 24% 116M 1s 2024-05-09T20:29:13.271 INFO:teuthology.orchestra.run.smithi063.stderr: 22800K .......... .......... .......... .......... .......... 24% 101M 1s 2024-05-09T20:29:13.271 INFO:teuthology.orchestra.run.smithi063.stderr: 22850K .......... .......... .......... .......... .......... 24% 109M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 22900K .......... .......... .......... .......... .......... 24% 113M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 22950K .......... .......... .......... .......... .......... 24% 119M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23000K .......... .......... .......... .......... .......... 24% 121M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23050K .......... .......... .......... .......... .......... 24% 75.2M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23100K .......... .......... .......... .......... .......... 24% 120M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23150K .......... .......... .......... .......... .......... 24% 126M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23200K .......... .......... .......... .......... .......... 25% 116M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23250K .......... .......... .......... .......... .......... 25% 94.2M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23300K .......... .......... .......... .......... .......... 25% 93.2M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23350K .......... .......... .......... .......... .......... 25% 111M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23400K .......... .......... .......... .......... .......... 25% 99.6M 1s 2024-05-09T20:29:13.272 INFO:teuthology.orchestra.run.smithi063.stderr: 23450K .......... .......... .......... .......... .......... 25% 110M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23500K .......... .......... .......... .......... .......... 25% 46.3M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23550K .......... .......... .......... .......... .......... 25% 74.2M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23600K .......... .......... .......... .......... .......... 25% 104M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23650K .......... .......... .......... .......... .......... 25% 104M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23700K .......... .......... .......... .......... .......... 25% 119M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23750K .......... .......... .......... .......... .......... 25% 96.1M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23800K .......... .......... .......... .......... .......... 25% 115M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23850K .......... .......... .......... .......... .......... 25% 104M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23900K .......... .......... .......... .......... .......... 25% 128M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 23950K .......... .......... .......... .......... .......... 25% 105M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 24000K .......... .......... .......... .......... .......... 25% 110M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 24050K .......... .......... .......... .......... .......... 25% 99.4M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 24100K .......... .......... .......... .......... .......... 25% 120M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 24150K .......... .......... .......... .......... .......... 26% 117M 1s 2024-05-09T20:29:13.273 INFO:teuthology.orchestra.run.smithi063.stderr: 24200K .......... .......... .......... .......... .......... 26% 102M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24250K .......... .......... .......... .......... .......... 26% 119M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24300K .......... .......... .......... .......... .......... 26% 115M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24350K .......... .......... .......... .......... .......... 26% 93.4M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24400K .......... .......... .......... .......... .......... 26% 119M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24450K .......... .......... .......... .......... .......... 26% 109M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24500K .......... .......... .......... .......... .......... 26% 98.6M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24550K .......... .......... .......... .......... .......... 26% 116M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24600K .......... .......... .......... .......... .......... 26% 112M 1s 2024-05-09T20:29:13.274 INFO:teuthology.orchestra.run.smithi063.stderr: 24650K .......... .......... .......... .......... .......... 26% 109M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 24700K .......... .......... .......... .......... .......... 26% 108M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 24750K .......... .......... .......... .......... .......... 26% 90.4M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 24800K .......... .......... .......... .......... .......... 26% 115M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 24850K .......... .......... .......... .......... .......... 26% 114M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 24900K .......... .......... .......... .......... .......... 26% 109M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 24950K .......... .......... .......... .......... .......... 26% 104M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 25000K .......... .......... .......... .......... .......... 26% 117M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 25050K .......... .......... .......... .......... .......... 27% 124M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 25100K .......... .......... .......... .......... .......... 27% 98.8M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 25150K .......... .......... .......... .......... .......... 27% 77.4M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 25200K .......... .......... .......... .......... .......... 27% 154M 1s 2024-05-09T20:29:13.275 INFO:teuthology.orchestra.run.smithi063.stderr: 25250K .......... .......... .......... .......... .......... 27% 110M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25300K .......... .......... .......... .......... .......... 27% 112M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25350K .......... .......... .......... .......... .......... 27% 110M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25400K .......... .......... .......... .......... .......... 27% 106M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25450K .......... .......... .......... .......... .......... 27% 122M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25500K .......... .......... .......... .......... .......... 27% 118M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25550K .......... .......... .......... .......... .......... 27% 82.6M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25600K .......... .......... .......... .......... .......... 27% 115M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25650K .......... .......... .......... .......... .......... 27% 103M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25700K .......... .......... .......... .......... .......... 27% 121M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25750K .......... .......... .......... .......... .......... 27% 104M 1s 2024-05-09T20:29:13.276 INFO:teuthology.orchestra.run.smithi063.stderr: 25800K .......... .......... .......... .......... .......... 27% 117M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 25850K .......... .......... .......... .......... .......... 27% 104M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 25900K .......... .......... .......... .......... .......... 27% 121M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 25950K .......... .......... .......... .......... .......... 27% 89.2M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 26000K .......... .......... .......... .......... .......... 28% 117M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 26050K .......... .......... .......... .......... .......... 28% 105M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 26100K .......... .......... .......... .......... .......... 28% 95.8M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 26150K .......... .......... .......... .......... .......... 28% 121M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 26200K .......... .......... .......... .......... .......... 28% 114M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 26250K .......... .......... .......... .......... .......... 28% 122M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 26300K .......... .......... .......... .......... .......... 28% 114M 1s 2024-05-09T20:29:13.277 INFO:teuthology.orchestra.run.smithi063.stderr: 26350K .......... .......... .......... .......... .......... 28% 90.6M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26400K .......... .......... .......... .......... .......... 28% 115M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26450K .......... .......... .......... .......... .......... 28% 112M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26500K .......... .......... .......... .......... .......... 28% 107M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26550K .......... .......... .......... .......... .......... 28% 83.4M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26600K .......... .......... .......... .......... .......... 28% 93.9M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26650K .......... .......... .......... .......... .......... 28% 109M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26700K .......... .......... .......... .......... .......... 28% 115M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26750K .......... .......... .......... .......... .......... 28% 89.9M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26800K .......... .......... .......... .......... .......... 28% 113M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26850K .......... .......... .......... .......... .......... 28% 97.5M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26900K .......... .......... .......... .......... .......... 29% 138M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 26950K .......... .......... .......... .......... .......... 29% 108M 1s 2024-05-09T20:29:13.278 INFO:teuthology.orchestra.run.smithi063.stderr: 27000K .......... .......... .......... .......... .......... 29% 104M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27050K .......... .......... .......... .......... .......... 29% 119M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27100K .......... .......... .......... .......... .......... 29% 109M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27150K .......... .......... .......... .......... .......... 29% 94.3M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27200K .......... .......... .......... .......... .......... 29% 115M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27250K .......... .......... .......... .......... .......... 29% 111M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27300K .......... .......... .......... .......... .......... 29% 121M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27350K .......... .......... .......... .......... .......... 29% 106M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27400K .......... .......... .......... .......... .......... 29% 120M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27450K .......... .......... .......... .......... .......... 29% 114M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27500K .......... .......... .......... .......... .......... 29% 91.1M 1s 2024-05-09T20:29:13.279 INFO:teuthology.orchestra.run.smithi063.stderr: 27550K .......... .......... .......... .......... .......... 29% 116M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 27600K .......... .......... .......... .......... .......... 29% 116M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 27650K .......... .......... .......... .......... .......... 29% 110M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 27700K .......... .......... .......... .......... .......... 29% 103M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 27750K .......... .......... .......... .......... .......... 29% 122M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 27800K .......... .......... .......... .......... .......... 29% 112M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 27850K .......... .......... .......... .......... .......... 30% 117M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 27900K .......... .......... .......... .......... .......... 30% 91.6M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 27950K .......... .......... .......... .......... .......... 30% 109M 1s 2024-05-09T20:29:13.280 INFO:teuthology.orchestra.run.smithi063.stderr: 28000K .......... .......... .......... .......... .......... 30% 113M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28050K .......... .......... .......... .......... .......... 30% 110M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28100K .......... .......... .......... .......... .......... 30% 111M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28150K .......... .......... .......... .......... .......... 30% 118M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28200K .......... .......... .......... .......... .......... 30% 88.9M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28250K .......... .......... .......... .......... .......... 30% 115M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28300K .......... .......... .......... .......... .......... 30% 112M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28350K .......... .......... .......... .......... .......... 30% 121M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28400K .......... .......... .......... .......... .......... 30% 95.3M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28450K .......... .......... .......... .......... .......... 30% 115M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28500K .......... .......... .......... .......... .......... 30% 123M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28550K .......... .......... .......... .......... .......... 30% 118M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28600K .......... .......... .......... .......... .......... 30% 88.8M 1s 2024-05-09T20:29:13.281 INFO:teuthology.orchestra.run.smithi063.stderr: 28650K .......... .......... .......... .......... .......... 30% 113M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 28700K .......... .......... .......... .......... .......... 30% 115M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 28750K .......... .......... .......... .......... .......... 31% 115M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 28800K .......... .......... .......... .......... .......... 31% 97.0M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 28850K .......... .......... .......... .......... .......... 31% 135M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 28900K .......... .......... .......... .......... .......... 31% 98.2M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 28950K .......... .......... .......... .......... .......... 31% 119M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 29000K .......... .......... .......... .......... .......... 31% 92.8M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 29050K .......... .......... .......... .......... .......... 31% 111M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 29100K .......... .......... .......... .......... .......... 31% 118M 1s 2024-05-09T20:29:13.282 INFO:teuthology.orchestra.run.smithi063.stderr: 29150K .......... .......... .......... .......... .......... 31% 56.7M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29200K .......... .......... .......... .......... .......... 31% 72.3M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29250K .......... .......... .......... .......... .......... 31% 114M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29300K .......... .......... .......... .......... .......... 31% 126M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29350K .......... .......... .......... .......... .......... 31% 110M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29400K .......... .......... .......... .......... .......... 31% 87.8M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29450K .......... .......... .......... .......... .......... 31% 118M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29500K .......... .......... .......... .......... .......... 31% 122M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29550K .......... .......... .......... .......... .......... 31% 107M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29600K .......... .......... .......... .......... .......... 31% 106M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29650K .......... .......... .......... .......... .......... 31% 112M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29700K .......... .......... .......... .......... .......... 32% 112M 1s 2024-05-09T20:29:13.283 INFO:teuthology.orchestra.run.smithi063.stderr: 29750K .......... .......... .......... .......... .......... 32% 113M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 29800K .......... .......... .......... .......... .......... 32% 88.2M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 29850K .......... .......... .......... .......... .......... 32% 118M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 29900K .......... .......... .......... .......... .......... 32% 112M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 29950K .......... .......... .......... .......... .......... 32% 113M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 30000K .......... .......... .......... .......... .......... 32% 109M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 30050K .......... .......... .......... .......... .......... 32% 117M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 30100K .......... .......... .......... .......... .......... 32% 107M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 30150K .......... .......... .......... .......... .......... 32% 120M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 30200K .......... .......... .......... .......... .......... 32% 85.4M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 30250K .......... .......... .......... .......... .......... 32% 115M 1s 2024-05-09T20:29:13.284 INFO:teuthology.orchestra.run.smithi063.stderr: 30300K .......... .......... .......... .......... .......... 32% 128M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30350K .......... .......... .......... .......... .......... 32% 109M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30400K .......... .......... .......... .......... .......... 32% 112M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30450K .......... .......... .......... .......... .......... 32% 115M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30500K .......... .......... .......... .......... .......... 32% 111M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30550K .......... .......... .......... .......... .......... 32% 106M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30600K .......... .......... .......... .......... .......... 32% 87.6M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30650K .......... .......... .......... .......... .......... 33% 121M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30700K .......... .......... .......... .......... .......... 33% 111M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30750K .......... .......... .......... .......... .......... 33% 121M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30800K .......... .......... .......... .......... .......... 33% 99.0M 1s 2024-05-09T20:29:13.285 INFO:teuthology.orchestra.run.smithi063.stderr: 30850K .......... .......... .......... .......... .......... 33% 103M 1s 2024-05-09T20:29:13.286 INFO:teuthology.orchestra.run.smithi063.stderr: 30900K .......... .......... .......... .......... .......... 33% 119M 1s 2024-05-09T20:29:13.286 INFO:teuthology.orchestra.run.smithi063.stderr: 30950K .......... .......... .......... .......... .......... 33% 133M 1s 2024-05-09T20:29:13.286 INFO:teuthology.orchestra.run.smithi063.stderr: 31000K .......... .......... .......... .......... .......... 33% 82.5M 1s 2024-05-09T20:29:13.286 INFO:teuthology.orchestra.run.smithi063.stderr: 31050K .......... .......... .......... .......... .......... 33% 115M 1s 2024-05-09T20:29:13.286 INFO:teuthology.orchestra.run.smithi063.stderr: 31100K .......... .......... .......... .......... .......... 33% 109M 1s 2024-05-09T20:29:13.286 INFO:teuthology.orchestra.run.smithi063.stderr: 31150K .......... .......... .......... .......... .......... 33% 121M 1s 2024-05-09T20:29:13.286 INFO:teuthology.orchestra.run.smithi063.stderr: 31200K .......... .......... .......... .......... .......... 33% 107M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31250K .......... .......... .......... .......... .......... 33% 112M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31300K .......... .......... .......... .......... .......... 33% 115M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31350K .......... .......... .......... .......... .......... 33% 114M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31400K .......... .......... .......... .......... .......... 33% 94.7M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31450K .......... .......... .......... .......... .......... 33% 112M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31500K .......... .......... .......... .......... .......... 33% 120M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31550K .......... .......... .......... .......... .......... 34% 114M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31600K .......... .......... .......... .......... .......... 34% 100M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31650K .......... .......... .......... .......... .......... 34% 120M 1s 2024-05-09T20:29:13.287 INFO:teuthology.orchestra.run.smithi063.stderr: 31700K .......... .......... .......... .......... .......... 34% 119M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 31750K .......... .......... .......... .......... .......... 34% 112M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 31800K .......... .......... .......... .......... .......... 34% 87.1M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 31850K .......... .......... .......... .......... .......... 34% 119M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 31900K .......... .......... .......... .......... .......... 34% 122M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 31950K .......... .......... .......... .......... .......... 34% 109M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 32000K .......... .......... .......... .......... .......... 34% 104M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 32050K .......... .......... .......... .......... .......... 34% 115M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 32100K .......... .......... .......... .......... .......... 34% 113M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 32150K .......... .......... .......... .......... .......... 34% 98.5M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 32200K .......... .......... .......... .......... .......... 34% 109M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 32250K .......... .......... .......... .......... .......... 34% 112M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 32300K .......... .......... .......... .......... .......... 34% 109M 1s 2024-05-09T20:29:13.288 INFO:teuthology.orchestra.run.smithi063.stderr: 32350K .......... .......... .......... .......... .......... 34% 107M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32400K .......... .......... .......... .......... .......... 34% 116M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32450K .......... .......... .......... .......... .......... 34% 119M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32500K .......... .......... .......... .......... .......... 35% 116M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32550K .......... .......... .......... .......... .......... 35% 90.6M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32600K .......... .......... .......... .......... .......... 35% 112M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32650K .......... .......... .......... .......... .......... 35% 115M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32700K .......... .......... .......... .......... .......... 35% 44.6M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32750K .......... .......... .......... .......... .......... 35% 113M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32800K .......... .......... .......... .......... .......... 35% 118M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32850K .......... .......... .......... .......... .......... 35% 95.2M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32900K .......... .......... .......... .......... .......... 35% 133M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 32950K .......... .......... .......... .......... .......... 35% 89.7M 1s 2024-05-09T20:29:13.289 INFO:teuthology.orchestra.run.smithi063.stderr: 33000K .......... .......... .......... .......... .......... 35% 119M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33050K .......... .......... .......... .......... .......... 35% 103M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33100K .......... .......... .......... .......... .......... 35% 131M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33150K .......... .......... .......... .......... .......... 35% 103M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33200K .......... .......... .......... .......... .......... 35% 114M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33250K .......... .......... .......... .......... .......... 35% 123M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33300K .......... .......... .......... .......... .......... 35% 102M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33350K .......... .......... .......... .......... .......... 35% 94.7M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33400K .......... .......... .......... .......... .......... 36% 115M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33450K .......... .......... .......... .......... .......... 36% 110M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33500K .......... .......... .......... .......... .......... 36% 118M 1s 2024-05-09T20:29:13.290 INFO:teuthology.orchestra.run.smithi063.stderr: 33550K .......... .......... .......... .......... .......... 36% 100M 1s 2024-05-09T20:29:13.291 INFO:teuthology.orchestra.run.smithi063.stderr: 33600K .......... .......... .......... .......... .......... 36% 114M 1s 2024-05-09T20:29:13.291 INFO:teuthology.orchestra.run.smithi063.stderr: 33650K .......... .......... .......... .......... .......... 36% 113M 1s 2024-05-09T20:29:13.291 INFO:teuthology.orchestra.run.smithi063.stderr: 33700K .......... .......... .......... .......... .......... 36% 117M 1s 2024-05-09T20:29:13.291 INFO:teuthology.orchestra.run.smithi063.stderr: 33750K .......... .......... .......... .......... .......... 36% 92.3M 1s 2024-05-09T20:29:13.291 INFO:teuthology.orchestra.run.smithi063.stderr: 33800K .......... .......... .......... .......... .......... 36% 106M 1s 2024-05-09T20:29:13.291 INFO:teuthology.orchestra.run.smithi063.stderr: 33850K .......... .......... .......... .......... .......... 36% 41.7M 1s 2024-05-09T20:29:13.291 INFO:teuthology.orchestra.run.smithi063.stderr: 33900K .......... .......... .......... .......... .......... 36% 120M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 33950K .......... .......... .......... .......... .......... 36% 99.8M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 34000K .......... .......... .......... .......... .......... 36% 123M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 34050K .......... .......... .......... .......... .......... 36% 117M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 34100K .......... .......... .......... .......... .......... 36% 110M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 34150K .......... .......... .......... .......... .......... 36% 70.3M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 34200K .......... .......... .......... .......... .......... 36% 108M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 34250K .......... .......... .......... .......... .......... 36% 194M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 34300K .......... .......... .......... .......... .......... 36% 110M 1s 2024-05-09T20:29:13.292 INFO:teuthology.orchestra.run.smithi063.stderr: 34350K .......... .......... .......... .......... .......... 37% 106M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34400K .......... .......... .......... .......... .......... 37% 117M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34450K .......... .......... .......... .......... .......... 37% 110M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34500K .......... .......... .......... .......... .......... 37% 120M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34550K .......... .......... .......... .......... .......... 37% 85.3M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34600K .......... .......... .......... .......... .......... 37% 99.1M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34650K .......... .......... .......... .......... .......... 37% 187M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34700K .......... .......... .......... .......... .......... 37% 87.0M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34750K .......... .......... .......... .......... .......... 37% 107M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34800K .......... .......... .......... .......... .......... 37% 118M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34850K .......... .......... .......... .......... .......... 37% 114M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34900K .......... .......... .......... .......... .......... 37% 107M 1s 2024-05-09T20:29:13.293 INFO:teuthology.orchestra.run.smithi063.stderr: 34950K .......... .......... .......... .......... .......... 37% 113M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35000K .......... .......... .......... .......... .......... 37% 110M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35050K .......... .......... .......... .......... .......... 37% 115M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35100K .......... .......... .......... .......... .......... 37% 86.4M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35150K .......... .......... .......... .......... .......... 37% 124M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35200K .......... .......... .......... .......... .......... 37% 109M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35250K .......... .......... .......... .......... .......... 37% 108M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35300K .......... .......... .......... .......... .......... 38% 113M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35350K .......... .......... .......... .......... .......... 38% 107M 1s 2024-05-09T20:29:13.294 INFO:teuthology.orchestra.run.smithi063.stderr: 35400K .......... .......... .......... .......... .......... 38% 117M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35450K .......... .......... .......... .......... .......... 38% 103M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35500K .......... .......... .......... .......... .......... 38% 86.6M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35550K .......... .......... .......... .......... .......... 38% 124M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35600K .......... .......... .......... .......... .......... 38% 114M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35650K .......... .......... .......... .......... .......... 38% 108M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35700K .......... .......... .......... .......... .......... 38% 114M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35750K .......... .......... .......... .......... .......... 38% 114M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35800K .......... .......... .......... .......... .......... 38% 112M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35850K .......... .......... .......... .......... .......... 38% 117M 1s 2024-05-09T20:29:13.295 INFO:teuthology.orchestra.run.smithi063.stderr: 35900K .......... .......... .......... .......... .......... 38% 89.6M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 35950K .......... .......... .......... .......... .......... 38% 112M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36000K .......... .......... .......... .......... .......... 38% 121M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36050K .......... .......... .......... .......... .......... 38% 116M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36100K .......... .......... .......... .......... .......... 38% 107M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36150K .......... .......... .......... .......... .......... 38% 110M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36200K .......... .......... .......... .......... .......... 39% 116M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36250K .......... .......... .......... .......... .......... 39% 112M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36300K .......... .......... .......... .......... .......... 39% 92.5M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36350K .......... .......... .......... .......... .......... 39% 102M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36400K .......... .......... .......... .......... .......... 39% 84.6M 1s 2024-05-09T20:29:13.296 INFO:teuthology.orchestra.run.smithi063.stderr: 36450K .......... .......... .......... .......... .......... 39% 116M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36500K .......... .......... .......... .......... .......... 39% 105M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36550K .......... .......... .......... .......... .......... 39% 117M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36600K .......... .......... .......... .......... .......... 39% 111M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36650K .......... .......... .......... .......... .......... 39% 116M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36700K .......... .......... .......... .......... .......... 39% 87.4M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36750K .......... .......... .......... .......... .......... 39% 124M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36800K .......... .......... .......... .......... .......... 39% 61.8M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36850K .......... .......... .......... .......... .......... 39% 192M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36900K .......... .......... .......... .......... .......... 39% 172M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 36950K .......... .......... .......... .......... .......... 39% 119M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 37000K .......... .......... .......... .......... .......... 39% 121M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 37050K .......... .......... .......... .......... .......... 39% 104M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 37100K .......... .......... .......... .......... .......... 39% 95.4M 1s 2024-05-09T20:29:13.297 INFO:teuthology.orchestra.run.smithi063.stderr: 37150K .......... .......... .......... .......... .......... 40% 121M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37200K .......... .......... .......... .......... .......... 40% 112M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37250K .......... .......... .......... .......... .......... 40% 118M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37300K .......... .......... .......... .......... .......... 40% 108M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37350K .......... .......... .......... .......... .......... 40% 111M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37400K .......... .......... .......... .......... .......... 40% 111M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37450K .......... .......... .......... .......... .......... 40% 119M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37500K .......... .......... .......... .......... .......... 40% 91.7M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37550K .......... .......... .......... .......... .......... 40% 114M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37600K .......... .......... .......... .......... .......... 40% 122M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37650K .......... .......... .......... .......... .......... 40% 109M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37700K .......... .......... .......... .......... .......... 40% 111M 1s 2024-05-09T20:29:13.298 INFO:teuthology.orchestra.run.smithi063.stderr: 37750K .......... .......... .......... .......... .......... 40% 113M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 37800K .......... .......... .......... .......... .......... 40% 116M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 37850K .......... .......... .......... .......... .......... 40% 117M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 37900K .......... .......... .......... .......... .......... 40% 88.4M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 37950K .......... .......... .......... .......... .......... 40% 117M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38000K .......... .......... .......... .......... .......... 40% 117M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38050K .......... .......... .......... .......... .......... 41% 114M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38100K .......... .......... .......... .......... .......... 41% 98.3M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38150K .......... .......... .......... .......... .......... 41% 115M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38200K .......... .......... .......... .......... .......... 41% 122M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38250K .......... .......... .......... .......... .......... 41% 110M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38300K .......... .......... .......... .......... .......... 41% 95.7M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38350K .......... .......... .......... .......... .......... 41% 111M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38400K .......... .......... .......... .......... .......... 41% 115M 1s 2024-05-09T20:29:13.299 INFO:teuthology.orchestra.run.smithi063.stderr: 38450K .......... .......... .......... .......... .......... 41% 110M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38500K .......... .......... .......... .......... .......... 41% 107M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38550K .......... .......... .......... .......... .......... 41% 112M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38600K .......... .......... .......... .......... .......... 41% 125M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38650K .......... .......... .......... .......... .......... 41% 103M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38700K .......... .......... .......... .......... .......... 41% 86.5M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38750K .......... .......... .......... .......... .......... 41% 114M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38800K .......... .......... .......... .......... .......... 41% 105M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38850K .......... .......... .......... .......... .......... 41% 121M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38900K .......... .......... .......... .......... .......... 41% 105M 1s 2024-05-09T20:29:13.300 INFO:teuthology.orchestra.run.smithi063.stderr: 38950K .......... .......... .......... .......... .......... 41% 112M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39000K .......... .......... .......... .......... .......... 42% 117M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39050K .......... .......... .......... .......... .......... 42% 118M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39100K .......... .......... .......... .......... .......... 42% 94.6M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39150K .......... .......... .......... .......... .......... 42% 113M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39200K .......... .......... .......... .......... .......... 42% 113M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39250K .......... .......... .......... .......... .......... 42% 110M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39300K .......... .......... .......... .......... .......... 42% 119M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39350K .......... .......... .......... .......... .......... 42% 115M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39400K .......... .......... .......... .......... .......... 42% 90.8M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39450K .......... .......... .......... .......... .......... 42% 47.8M 1s 2024-05-09T20:29:13.301 INFO:teuthology.orchestra.run.smithi063.stderr: 39500K .......... .......... .......... .......... .......... 42% 107M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39550K .......... .......... .......... .......... .......... 42% 115M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39600K .......... .......... .......... .......... .......... 42% 107M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39650K .......... .......... .......... .......... .......... 42% 92.4M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39700K .......... .......... .......... .......... .......... 42% 120M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39750K .......... .......... .......... .......... .......... 42% 116M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39800K .......... .......... .......... .......... .......... 42% 100M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39850K .......... .......... .......... .......... .......... 42% 96.8M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39900K .......... .......... .......... .......... .......... 43% 115M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 39950K .......... .......... .......... .......... .......... 43% 115M 1s 2024-05-09T20:29:13.302 INFO:teuthology.orchestra.run.smithi063.stderr: 40000K .......... .......... .......... .......... .......... 43% 111M 1s 2024-05-09T20:29:13.303 INFO:teuthology.orchestra.run.smithi063.stderr: 40050K .......... .......... .......... .......... .......... 43% 87.8M 1s 2024-05-09T20:29:13.303 INFO:teuthology.orchestra.run.smithi063.stderr: 40100K .......... .......... .......... .......... .......... 43% 120M 1s 2024-05-09T20:29:13.303 INFO:teuthology.orchestra.run.smithi063.stderr: 40150K .......... .......... .......... .......... .......... 43% 112M 1s 2024-05-09T20:29:13.303 INFO:teuthology.orchestra.run.smithi063.stderr: 40200K .......... .......... .......... .......... .......... 43% 113M 1s 2024-05-09T20:29:13.303 INFO:teuthology.orchestra.run.smithi063.stderr: 40250K .......... .......... .......... .......... .......... 43% 110M 1s 2024-05-09T20:29:13.303 INFO:teuthology.orchestra.run.smithi063.stderr: 40300K .......... .......... .......... .......... .......... 43% 108M 1s 2024-05-09T20:29:13.303 INFO:teuthology.orchestra.run.smithi063.stderr: 40350K .......... .......... .......... .......... .......... 43% 111M 1s 2024-05-09T20:29:13.303 INFO:teuthology.orchestra.run.smithi063.stderr: 40400K .......... .......... .......... .......... .......... 43% 104M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40450K .......... .......... .......... .......... .......... 43% 106M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40500K .......... .......... .......... .......... .......... 43% 110M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40550K .......... .......... .......... .......... .......... 43% 126M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40600K .......... .......... .......... .......... .......... 43% 106M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40650K .......... .......... .......... .......... .......... 43% 118M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40700K .......... .......... .......... .......... .......... 43% 108M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40750K .......... .......... .......... .......... .......... 43% 110M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40800K .......... .......... .......... .......... .......... 43% 83.2M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40850K .......... .......... .......... .......... .......... 44% 116M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40900K .......... .......... .......... .......... .......... 44% 116M 1s 2024-05-09T20:29:13.304 INFO:teuthology.orchestra.run.smithi063.stderr: 40950K .......... .......... .......... .......... .......... 44% 114M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41000K .......... .......... .......... .......... .......... 44% 105M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41050K .......... .......... .......... .......... .......... 44% 109M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41100K .......... .......... .......... .......... .......... 44% 127M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41150K .......... .......... .......... .......... .......... 44% 113M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41200K .......... .......... .......... .......... .......... 44% 89.0M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41250K .......... .......... .......... .......... .......... 44% 118M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41300K .......... .......... .......... .......... .......... 44% 110M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41350K .......... .......... .......... .......... .......... 44% 116M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41400K .......... .......... .......... .......... .......... 44% 105M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41450K .......... .......... .......... .......... .......... 44% 118M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41500K .......... .......... .......... .......... .......... 44% 120M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41550K .......... .......... .......... .......... .......... 44% 108M 1s 2024-05-09T20:29:13.305 INFO:teuthology.orchestra.run.smithi063.stderr: 41600K .......... .......... .......... .......... .......... 44% 92.5M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 41650K .......... .......... .......... .......... .......... 44% 111M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 41700K .......... .......... .......... .......... .......... 44% 115M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 41750K .......... .......... .......... .......... .......... 44% 109M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 41800K .......... .......... .......... .......... .......... 45% 101M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 41850K .......... .......... .......... .......... .......... 45% 119M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 41900K .......... .......... .......... .......... .......... 45% 112M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 41950K .......... .......... .......... .......... .......... 45% 119M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 42000K .......... .......... .......... .......... .......... 45% 91.6M 1s 2024-05-09T20:29:13.306 INFO:teuthology.orchestra.run.smithi063.stderr: 42050K .......... .......... .......... .......... .......... 45% 107M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42100K .......... .......... .......... .......... .......... 45% 120M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42150K .......... .......... .......... .......... .......... 45% 110M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42200K .......... .......... .......... .......... .......... 45% 107M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42250K .......... .......... .......... .......... .......... 45% 130M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42300K .......... .......... .......... .......... .......... 45% 107M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42350K .......... .......... .......... .......... .......... 45% 119M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42400K .......... .......... .......... .......... .......... 45% 88.4M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42450K .......... .......... .......... .......... .......... 45% 115M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42500K .......... .......... .......... .......... .......... 45% 111M 1s 2024-05-09T20:29:13.307 INFO:teuthology.orchestra.run.smithi063.stderr: 42550K .......... .......... .......... .......... .......... 45% 113M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 42600K .......... .......... .......... .......... .......... 45% 114M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 42650K .......... .......... .......... .......... .......... 45% 106M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 42700K .......... .......... .......... .......... .......... 46% 121M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 42750K .......... .......... .......... .......... .......... 46% 113M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 42800K .......... .......... .......... .......... .......... 46% 89.8M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 42850K .......... .......... .......... .......... .......... 46% 123M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 42900K .......... .......... .......... .......... .......... 46% 109M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 42950K .......... .......... .......... .......... .......... 46% 45.9M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 43000K .......... .......... .......... .......... .......... 46% 115M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 43050K .......... .......... .......... .......... .......... 46% 121M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 43100K .......... .......... .......... .......... .......... 46% 88.1M 1s 2024-05-09T20:29:13.308 INFO:teuthology.orchestra.run.smithi063.stderr: 43150K .......... .......... .......... .......... .......... 46% 101M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43200K .......... .......... .......... .......... .......... 46% 117M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43250K .......... .......... .......... .......... .......... 46% 112M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43300K .......... .......... .......... .......... .......... 46% 65.0M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43350K .......... .......... .......... .......... .......... 46% 74.6M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43400K .......... .......... .......... .......... .......... 46% 101M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43450K .......... .......... .......... .......... .......... 46% 111M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43500K .......... .......... .......... .......... .......... 46% 91.3M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43550K .......... .......... .......... .......... .......... 46% 109M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43600K .......... .......... .......... .......... .......... 46% 109M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43650K .......... .......... .......... .......... .......... 47% 110M 1s 2024-05-09T20:29:13.309 INFO:teuthology.orchestra.run.smithi063.stderr: 43700K .......... .......... .......... .......... .......... 47% 99.1M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 43750K .......... .......... .......... .......... .......... 47% 111M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 43800K .......... .......... .......... .......... .......... 47% 107M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 43850K .......... .......... .......... .......... .......... 47% 110M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 43900K .......... .......... .......... .......... .......... 47% 96.8M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 43950K .......... .......... .......... .......... .......... 47% 118M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 44000K .......... .......... .......... .......... .......... 47% 117M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 44050K .......... .......... .......... .......... .......... 47% 127M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 44100K .......... .......... .......... .......... .......... 47% 107M 1s 2024-05-09T20:29:13.310 INFO:teuthology.orchestra.run.smithi063.stderr: 44150K .......... .......... .......... .......... .......... 47% 116M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44200K .......... .......... .......... .......... .......... 47% 119M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44250K .......... .......... .......... .......... .......... 47% 115M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44300K .......... .......... .......... .......... .......... 47% 75.5M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44350K .......... .......... .......... .......... .......... 47% 109M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44400K .......... .......... .......... .......... .......... 47% 112M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44450K .......... .......... .......... .......... .......... 47% 113M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44500K .......... .......... .......... .......... .......... 47% 64.0M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44550K .......... .......... .......... .......... .......... 48% 156M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44600K .......... .......... .......... .......... .......... 48% 152M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44650K .......... .......... .......... .......... .......... 48% 127M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44700K .......... .......... .......... .......... .......... 48% 95.6M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44750K .......... .......... .......... .......... .......... 48% 121M 1s 2024-05-09T20:29:13.311 INFO:teuthology.orchestra.run.smithi063.stderr: 44800K .......... .......... .......... .......... .......... 48% 106M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 44850K .......... .......... .......... .......... .......... 48% 126M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 44900K .......... .......... .......... .......... .......... 48% 107M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 44950K .......... .......... .......... .......... .......... 48% 116M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45000K .......... .......... .......... .......... .......... 48% 118M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45050K .......... .......... .......... .......... .......... 48% 92.0M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45100K .......... .......... .......... .......... .......... 48% 114M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45150K .......... .......... .......... .......... .......... 48% 112M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45200K .......... .......... .......... .......... .......... 48% 114M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45250K .......... .......... .......... .......... .......... 48% 109M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45300K .......... .......... .......... .......... .......... 48% 113M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45350K .......... .......... .......... .......... .......... 48% 121M 1s 2024-05-09T20:29:13.312 INFO:teuthology.orchestra.run.smithi063.stderr: 45400K .......... .......... .......... .......... .......... 48% 111M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45450K .......... .......... .......... .......... .......... 48% 88.8M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45500K .......... .......... .......... .......... .......... 49% 116M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45550K .......... .......... .......... .......... .......... 49% 107M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45600K .......... .......... .......... .......... .......... 49% 118M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45650K .......... .......... .......... .......... .......... 49% 110M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45700K .......... .......... .......... .......... .......... 49% 104M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45750K .......... .......... .......... .......... .......... 49% 126M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45800K .......... .......... .......... .......... .......... 49% 94.7M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45850K .......... .......... .......... .......... .......... 49% 100M 1s 2024-05-09T20:29:13.313 INFO:teuthology.orchestra.run.smithi063.stderr: 45900K .......... .......... .......... .......... .......... 49% 127M 1s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 45950K .......... .......... .......... .......... .......... 49% 116M 1s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46000K .......... .......... .......... .......... .......... 49% 116M 1s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46050K .......... .......... .......... .......... .......... 49% 103M 1s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46100K .......... .......... .......... .......... .......... 49% 118M 0s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46150K .......... .......... .......... .......... .......... 49% 107M 0s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46200K .......... .......... .......... .......... .......... 49% 117M 0s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46250K .......... .......... .......... .......... .......... 49% 91.3M 0s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46300K .......... .......... .......... .......... .......... 49% 110M 0s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46350K .......... .......... .......... .......... .......... 49% 115M 0s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46400K .......... .......... .......... .......... .......... 49% 112M 0s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46450K .......... .......... .......... .......... .......... 50% 102M 0s 2024-05-09T20:29:13.314 INFO:teuthology.orchestra.run.smithi063.stderr: 46500K .......... .......... .......... .......... .......... 50% 124M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46550K .......... .......... .......... .......... .......... 50% 111M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46600K .......... .......... .......... .......... .......... 50% 118M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46650K .......... .......... .......... .......... .......... 50% 90.7M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46700K .......... .......... .......... .......... .......... 50% 116M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46750K .......... .......... .......... .......... .......... 50% 113M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46800K .......... .......... .......... .......... .......... 50% 116M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46850K .......... .......... .......... .......... .......... 50% 107M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46900K .......... .......... .......... .......... .......... 50% 109M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 46950K .......... .......... .......... .......... .......... 50% 113M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 47000K .......... .......... .......... .......... .......... 50% 120M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 47050K .......... .......... .......... .......... .......... 50% 86.9M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 47100K .......... .......... .......... .......... .......... 50% 120M 0s 2024-05-09T20:29:13.315 INFO:teuthology.orchestra.run.smithi063.stderr: 47150K .......... .......... .......... .......... .......... 50% 107M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47200K .......... .......... .......... .......... .......... 50% 121M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47250K .......... .......... .......... .......... .......... 50% 113M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47300K .......... .......... .......... .......... .......... 50% 117M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47350K .......... .......... .......... .......... .......... 51% 87.6M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47400K .......... .......... .......... .......... .......... 51% 119M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47450K .......... .......... .......... .......... .......... 51% 109M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47500K .......... .......... .......... .......... .......... 51% 117M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47550K .......... .......... .......... .......... .......... 51% 104M 0s 2024-05-09T20:29:13.316 INFO:teuthology.orchestra.run.smithi063.stderr: 47600K .......... .......... .......... .......... .......... 51% 114M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 47650K .......... .......... .......... .......... .......... 51% 91.4M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 47700K .......... .......... .......... .......... .......... 51% 119M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 47750K .......... .......... .......... .......... .......... 51% 90.2M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 47800K .......... .......... .......... .......... .......... 51% 113M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 47850K .......... .......... .......... .......... .......... 51% 107M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 47900K .......... .......... .......... .......... .......... 51% 111M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 47950K .......... .......... .......... .......... .......... 51% 109M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 48000K .......... .......... .......... .......... .......... 51% 112M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 48050K .......... .......... .......... .......... .......... 51% 117M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 48100K .......... .......... .......... .......... .......... 51% 116M 0s 2024-05-09T20:29:13.317 INFO:teuthology.orchestra.run.smithi063.stderr: 48150K .......... .......... .......... .......... .......... 51% 89.1M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48200K .......... .......... .......... .......... .......... 51% 119M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48250K .......... .......... .......... .......... .......... 51% 115M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48300K .......... .......... .......... .......... .......... 52% 106M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48350K .......... .......... .......... .......... .......... 52% 111M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48400K .......... .......... .......... .......... .......... 52% 118M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48450K .......... .......... .......... .......... .......... 52% 115M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48500K .......... .......... .......... .......... .......... 52% 94.2M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48550K .......... .......... .......... .......... .......... 52% 110M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48600K .......... .......... .......... .......... .......... 52% 112M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48650K .......... .......... .......... .......... .......... 52% 117M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48700K .......... .......... .......... .......... .......... 52% 108M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48750K .......... .......... .......... .......... .......... 52% 117M 0s 2024-05-09T20:29:13.318 INFO:teuthology.orchestra.run.smithi063.stderr: 48800K .......... .......... .......... .......... .......... 52% 115M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 48850K .......... .......... .......... .......... .......... 52% 112M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 48900K .......... .......... .......... .......... .......... 52% 89.9M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 48950K .......... .......... .......... .......... .......... 52% 113M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 49000K .......... .......... .......... .......... .......... 52% 116M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 49050K .......... .......... .......... .......... .......... 52% 111M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 49100K .......... .......... .......... .......... .......... 52% 111M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 49150K .......... .......... .......... .......... .......... 52% 104M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 49200K .......... .......... .......... .......... .......... 53% 125M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 49250K .......... .......... .......... .......... .......... 53% 101M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 49300K .......... .......... .......... .......... .......... 53% 94.3M 0s 2024-05-09T20:29:13.319 INFO:teuthology.orchestra.run.smithi063.stderr: 49350K .......... .......... .......... .......... .......... 53% 112M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49400K .......... .......... .......... .......... .......... 53% 113M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49450K .......... .......... .......... .......... .......... 53% 124M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49500K .......... .......... .......... .......... .......... 53% 99.6M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49550K .......... .......... .......... .......... .......... 53% 124M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49600K .......... .......... .......... .......... .......... 53% 104M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49650K .......... .......... .......... .......... .......... 53% 104M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49700K .......... .......... .......... .......... .......... 53% 46.0M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49750K .......... .......... .......... .......... .......... 53% 99.9M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49800K .......... .......... .......... .......... .......... 53% 141M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49850K .......... .......... .......... .......... .......... 53% 116M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49900K .......... .......... .......... .......... .......... 53% 103M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 49950K .......... .......... .......... .......... .......... 53% 110M 0s 2024-05-09T20:29:13.320 INFO:teuthology.orchestra.run.smithi063.stderr: 50000K .......... .......... .......... .......... .......... 53% 119M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50050K .......... .......... .......... .......... .......... 53% 113M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50100K .......... .......... .......... .......... .......... 53% 76.2M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50150K .......... .......... .......... .......... .......... 54% 102M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50200K .......... .......... .......... .......... .......... 54% 110M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50250K .......... .......... .......... .......... .......... 54% 102M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50300K .......... .......... .......... .......... .......... 54% 138M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50350K .......... .......... .......... .......... .......... 54% 116M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50400K .......... .......... .......... .......... .......... 54% 74.9M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50450K .......... .......... .......... .......... .......... 54% 142M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50500K .......... .......... .......... .......... .......... 54% 109M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50550K .......... .......... .......... .......... .......... 54% 113M 0s 2024-05-09T20:29:13.321 INFO:teuthology.orchestra.run.smithi063.stderr: 50600K .......... .......... .......... .......... .......... 54% 96.1M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 50650K .......... .......... .......... .......... .......... 54% 118M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 50700K .......... .......... .......... .......... .......... 54% 109M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 50750K .......... .......... .......... .......... .......... 54% 116M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 50800K .......... .......... .......... .......... .......... 54% 90.8M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 50850K .......... .......... .......... .......... .......... 54% 119M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 50900K .......... .......... .......... .......... .......... 54% 120M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 50950K .......... .......... .......... .......... .......... 54% 103M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 51000K .......... .......... .......... .......... .......... 54% 114M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 51050K .......... .......... .......... .......... .......... 55% 117M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 51100K .......... .......... .......... .......... .......... 55% 110M 0s 2024-05-09T20:29:13.322 INFO:teuthology.orchestra.run.smithi063.stderr: 51150K .......... .......... .......... .......... .......... 55% 91.7M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51200K .......... .......... .......... .......... .......... 55% 113M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51250K .......... .......... .......... .......... .......... 55% 108M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51300K .......... .......... .......... .......... .......... 55% 117M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51350K .......... .......... .......... .......... .......... 55% 106M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51400K .......... .......... .......... .......... .......... 55% 116M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51450K .......... .......... .......... .......... .......... 55% 94.4M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51500K .......... .......... .......... .......... .......... 55% 116M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51550K .......... .......... .......... .......... .......... 55% 114M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51600K .......... .......... .......... .......... .......... 55% 113M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51650K .......... .......... .......... .......... .......... 55% 101M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51700K .......... .......... .......... .......... .......... 55% 127M 0s 2024-05-09T20:29:13.323 INFO:teuthology.orchestra.run.smithi063.stderr: 51750K .......... .......... .......... .......... .......... 55% 109M 0s 2024-05-09T20:29:13.324 INFO:teuthology.orchestra.run.smithi063.stderr: 51800K .......... .......... .......... .......... .......... 55% 109M 0s 2024-05-09T20:29:13.324 INFO:teuthology.orchestra.run.smithi063.stderr: 51850K .......... .......... .......... .......... .......... 55% 93.7M 0s 2024-05-09T20:29:13.324 INFO:teuthology.orchestra.run.smithi063.stderr: 51900K .......... .......... .......... .......... .......... 55% 109M 0s 2024-05-09T20:29:13.325 INFO:teuthology.orchestra.run.smithi063.stderr: 51950K .......... .......... .......... .......... .......... 55% 121M 0s 2024-05-09T20:29:13.325 INFO:teuthology.orchestra.run.smithi063.stderr: 52000K .......... .......... .......... .......... .......... 56% 106M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52050K .......... .......... .......... .......... .......... 56% 114M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52100K .......... .......... .......... .......... .......... 56% 130M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52150K .......... .......... .......... .......... .......... 56% 91.3M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52200K .......... .......... .......... .......... .......... 56% 111M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52250K .......... .......... .......... .......... .......... 56% 120M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52300K .......... .......... .......... .......... .......... 56% 132M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52350K .......... .......... .......... .......... .......... 56% 114M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52400K .......... .......... .......... .......... .......... 56% 80.9M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52450K .......... .......... .......... .......... .......... 56% 119M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52500K .......... .......... .......... .......... .......... 56% 114M 0s 2024-05-09T20:29:13.326 INFO:teuthology.orchestra.run.smithi063.stderr: 52550K .......... .......... .......... .......... .......... 56% 133M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 52600K .......... .......... .......... .......... .......... 56% 102M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 52650K .......... .......... .......... .......... .......... 56% 95.0M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 52700K .......... .......... .......... .......... .......... 56% 112M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 52750K .......... .......... .......... .......... .......... 56% 117M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 52800K .......... .......... .......... .......... .......... 56% 96.4M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 52850K .......... .......... .......... .......... .......... 56% 129M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 52900K .......... .......... .......... .......... .......... 56% 107M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 52950K .......... .......... .......... .......... .......... 57% 118M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 53000K .......... .......... .......... .......... .......... 57% 117M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 53050K .......... .......... .......... .......... .......... 57% 91.1M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 53100K .......... .......... .......... .......... .......... 57% 106M 0s 2024-05-09T20:29:13.327 INFO:teuthology.orchestra.run.smithi063.stderr: 53150K .......... .......... .......... .......... .......... 57% 48.6M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53200K .......... .......... .......... .......... .......... 57% 110M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53250K .......... .......... .......... .......... .......... 57% 115M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53300K .......... .......... .......... .......... .......... 57% 120M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53350K .......... .......... .......... .......... .......... 57% 89.5M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53400K .......... .......... .......... .......... .......... 57% 116M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53450K .......... .......... .......... .......... .......... 57% 109M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53500K .......... .......... .......... .......... .......... 57% 106M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53550K .......... .......... .......... .......... .......... 57% 107M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53600K .......... .......... .......... .......... .......... 57% 128M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53650K .......... .......... .......... .......... .......... 57% 104M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53700K .......... .......... .......... .......... .......... 57% 103M 0s 2024-05-09T20:29:13.328 INFO:teuthology.orchestra.run.smithi063.stderr: 53750K .......... .......... .......... .......... .......... 57% 90.2M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 53800K .......... .......... .......... .......... .......... 57% 122M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 53850K .......... .......... .......... .......... .......... 58% 109M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 53900K .......... .......... .......... .......... .......... 58% 115M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 53950K .......... .......... .......... .......... .......... 58% 101M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 54000K .......... .......... .......... .......... .......... 58% 97.9M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 54050K .......... .......... .......... .......... .......... 58% 122M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 54100K .......... .......... .......... .......... .......... 58% 114M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 54150K .......... .......... .......... .......... .......... 58% 107M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 54200K .......... .......... .......... .......... .......... 58% 110M 0s 2024-05-09T20:29:13.329 INFO:teuthology.orchestra.run.smithi063.stderr: 54250K .......... .......... .......... .......... .......... 58% 108M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54300K .......... .......... .......... .......... .......... 58% 118M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54350K .......... .......... .......... .......... .......... 58% 108M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54400K .......... .......... .......... .......... .......... 58% 96.6M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54450K .......... .......... .......... .......... .......... 58% 103M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54500K .......... .......... .......... .......... .......... 58% 114M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54550K .......... .......... .......... .......... .......... 58% 113M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54600K .......... .......... .......... .......... .......... 58% 45.6M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54650K .......... .......... .......... .......... .......... 58% 116M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54700K .......... .......... .......... .......... .......... 58% 110M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54750K .......... .......... .......... .......... .......... 58% 114M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54800K .......... .......... .......... .......... .......... 59% 87.3M 0s 2024-05-09T20:29:13.330 INFO:teuthology.orchestra.run.smithi063.stderr: 54850K .......... .......... .......... .......... .......... 59% 129M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 54900K .......... .......... .......... .......... .......... 59% 93.7M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 54950K .......... .......... .......... .......... .......... 59% 127M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55000K .......... .......... .......... .......... .......... 59% 121M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55050K .......... .......... .......... .......... .......... 59% 93.5M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55100K .......... .......... .......... .......... .......... 59% 155M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55150K .......... .......... .......... .......... .......... 59% 113M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55200K .......... .......... .......... .......... .......... 59% 75.3M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55250K .......... .......... .......... .......... .......... 59% 98.9M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55300K .......... .......... .......... .......... .......... 59% 121M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55350K .......... .......... .......... .......... .......... 59% 122M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55400K .......... .......... .......... .......... .......... 59% 111M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55450K .......... .......... .......... .......... .......... 59% 128M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55500K .......... .......... .......... .......... .......... 59% 122M 0s 2024-05-09T20:29:13.331 INFO:teuthology.orchestra.run.smithi063.stderr: 55550K .......... .......... .......... .......... .......... 59% 125M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 55600K .......... .......... .......... .......... .......... 59% 93.4M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 55650K .......... .......... .......... .......... .......... 59% 111M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 55700K .......... .......... .......... .......... .......... 60% 112M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 55750K .......... .......... .......... .......... .......... 60% 105M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 55800K .......... .......... .......... .......... .......... 60% 97.9M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 55850K .......... .......... .......... .......... .......... 60% 114M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 55900K .......... .......... .......... .......... .......... 60% 120M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 55950K .......... .......... .......... .......... .......... 60% 96.8M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 56000K .......... .......... .......... .......... .......... 60% 115M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 56050K .......... .......... .......... .......... .......... 60% 108M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 56100K .......... .......... .......... .......... .......... 60% 108M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 56150K .......... .......... .......... .......... .......... 60% 112M 0s 2024-05-09T20:29:13.332 INFO:teuthology.orchestra.run.smithi063.stderr: 56200K .......... .......... .......... .......... .......... 60% 82.0M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56250K .......... .......... .......... .......... .......... 60% 115M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56300K .......... .......... .......... .......... .......... 60% 116M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56350K .......... .......... .......... .......... .......... 60% 99.0M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56400K .......... .......... .......... .......... .......... 60% 111M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56450K .......... .......... .......... .......... .......... 60% 107M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56500K .......... .......... .......... .......... .......... 60% 122M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56550K .......... .......... .......... .......... .......... 60% 101M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56600K .......... .......... .......... .......... .......... 60% 109M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56650K .......... .......... .......... .......... .......... 61% 118M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56700K .......... .......... .......... .......... .......... 61% 93.0M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56750K .......... .......... .......... .......... .......... 61% 109M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56800K .......... .......... .......... .......... .......... 61% 126M 0s 2024-05-09T20:29:13.333 INFO:teuthology.orchestra.run.smithi063.stderr: 56850K .......... .......... .......... .......... .......... 61% 113M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 56900K .......... .......... .......... .......... .......... 61% 109M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 56950K .......... .......... .......... .......... .......... 61% 116M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57000K .......... .......... .......... .......... .......... 61% 110M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57050K .......... .......... .......... .......... .......... 61% 110M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57100K .......... .......... .......... .......... .......... 61% 94.5M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57150K .......... .......... .......... .......... .......... 61% 113M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57200K .......... .......... .......... .......... .......... 61% 111M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57250K .......... .......... .......... .......... .......... 61% 113M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57300K .......... .......... .......... .......... .......... 61% 88.5M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57350K .......... .......... .......... .......... .......... 61% 132M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57400K .......... .......... .......... .......... .......... 61% 101M 0s 2024-05-09T20:29:13.334 INFO:teuthology.orchestra.run.smithi063.stderr: 57450K .......... .......... .......... .......... .......... 61% 109M 0s 2024-05-09T20:29:13.335 INFO:teuthology.orchestra.run.smithi063.stderr: 57500K .......... .......... .......... .......... .......... 61% 110M 0s 2024-05-09T20:29:13.335 INFO:teuthology.orchestra.run.smithi063.stderr: 57550K .......... .......... .......... .......... .......... 62% 112M 0s 2024-05-09T20:29:13.335 INFO:teuthology.orchestra.run.smithi063.stderr: 57600K .......... .......... .......... .......... .......... 62% 120M 0s 2024-05-09T20:29:13.335 INFO:teuthology.orchestra.run.smithi063.stderr: 57650K .......... .......... .......... .......... .......... 62% 94.5M 0s 2024-05-09T20:29:13.335 INFO:teuthology.orchestra.run.smithi063.stderr: 57700K .......... .......... .......... .......... .......... 62% 115M 0s 2024-05-09T20:29:13.335 INFO:teuthology.orchestra.run.smithi063.stderr: 57750K .......... .......... .......... .......... .......... 62% 129M 0s 2024-05-09T20:29:13.335 INFO:teuthology.orchestra.run.smithi063.stderr: 57800K .......... .......... .......... .......... .......... 62% 113M 0s 2024-05-09T20:29:13.335 INFO:teuthology.orchestra.run.smithi063.stderr: 57850K .......... .......... .......... .......... .......... 62% 119M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 57900K .......... .......... .......... .......... .......... 62% 92.3M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 57950K .......... .......... .......... .......... .......... 62% 117M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58000K .......... .......... .......... .......... .......... 62% 115M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58050K .......... .......... .......... .......... .......... 62% 111M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58100K .......... .......... .......... .......... .......... 62% 108M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58150K .......... .......... .......... .......... .......... 62% 45.0M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58200K .......... .......... .......... .......... .......... 62% 139M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58250K .......... .......... .......... .......... .......... 62% 134M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58300K .......... .......... .......... .......... .......... 62% 105M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58350K .......... .......... .......... .......... .......... 62% 135M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58400K .......... .......... .......... .......... .......... 62% 134M 0s 2024-05-09T20:29:13.336 INFO:teuthology.orchestra.run.smithi063.stderr: 58450K .......... .......... .......... .......... .......... 62% 131M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58500K .......... .......... .......... .......... .......... 63% 110M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58550K .......... .......... .......... .......... .......... 63% 120M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58600K .......... .......... .......... .......... .......... 63% 110M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58650K .......... .......... .......... .......... .......... 63% 100M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58700K .......... .......... .......... .......... .......... 63% 123M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58750K .......... .......... .......... .......... .......... 63% 112M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58800K .......... .......... .......... .......... .......... 63% 101M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58850K .......... .......... .......... .......... .......... 63% 111M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58900K .......... .......... .......... .......... .......... 63% 107M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 58950K .......... .......... .......... .......... .......... 63% 117M 0s 2024-05-09T20:29:13.337 INFO:teuthology.orchestra.run.smithi063.stderr: 59000K .......... .......... .......... .......... .......... 63% 111M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59050K .......... .......... .......... .......... .......... 63% 112M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59100K .......... .......... .......... .......... .......... 63% 93.5M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59150K .......... .......... .......... .......... .......... 63% 118M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59200K .......... .......... .......... .......... .......... 63% 103M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59250K .......... .......... .......... .......... .......... 63% 107M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59300K .......... .......... .......... .......... .......... 63% 95.7M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59350K .......... .......... .......... .......... .......... 63% 149M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59400K .......... .......... .......... .......... .......... 63% 87.1M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59450K .......... .......... .......... .......... .......... 64% 128M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59500K .......... .......... .......... .......... .......... 64% 101M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59550K .......... .......... .......... .......... .......... 64% 142M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59600K .......... .......... .......... .......... .......... 64% 108M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59650K .......... .......... .......... .......... .......... 64% 112M 0s 2024-05-09T20:29:13.338 INFO:teuthology.orchestra.run.smithi063.stderr: 59700K .......... .......... .......... .......... .......... 64% 113M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 59750K .......... .......... .......... .......... .......... 64% 117M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 59800K .......... .......... .......... .......... .......... 64% 88.9M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 59850K .......... .......... .......... .......... .......... 64% 115M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 59900K .......... .......... .......... .......... .......... 64% 118M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 59950K .......... .......... .......... .......... .......... 64% 45.5M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 60000K .......... .......... .......... .......... .......... 64% 95.1M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 60050K .......... .......... .......... .......... .......... 64% 129M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 60100K .......... .......... .......... .......... .......... 64% 103M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 60150K .......... .......... .......... .......... .......... 64% 137M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 60200K .......... .......... .......... .......... .......... 64% 88.1M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 60250K .......... .......... .......... .......... .......... 64% 114M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 60300K .......... .......... .......... .......... .......... 64% 113M 0s 2024-05-09T20:29:13.339 INFO:teuthology.orchestra.run.smithi063.stderr: 60350K .......... .......... .......... .......... .......... 65% 119M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60400K .......... .......... .......... .......... .......... 65% 102M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60450K .......... .......... .......... .......... .......... 65% 108M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60500K .......... .......... .......... .......... .......... 65% 121M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60550K .......... .......... .......... .......... .......... 65% 104M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60600K .......... .......... .......... .......... .......... 65% 102M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60650K .......... .......... .......... .......... .......... 65% 90.0M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60700K .......... .......... .......... .......... .......... 65% 110M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60750K .......... .......... .......... .......... .......... 65% 113M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60800K .......... .......... .......... .......... .......... 65% 103M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60850K .......... .......... .......... .......... .......... 65% 106M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60900K .......... .......... .......... .......... .......... 65% 111M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 60950K .......... .......... .......... .......... .......... 65% 106M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 61000K .......... .......... .......... .......... .......... 65% 94.2M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 61050K .......... .......... .......... .......... .......... 65% 116M 0s 2024-05-09T20:29:13.340 INFO:teuthology.orchestra.run.smithi063.stderr: 61100K .......... .......... .......... .......... .......... 65% 115M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61150K .......... .......... .......... .......... .......... 65% 113M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61200K .......... .......... .......... .......... .......... 65% 103M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61250K .......... .......... .......... .......... .......... 65% 117M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61300K .......... .......... .......... .......... .......... 66% 119M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61350K .......... .......... .......... .......... .......... 66% 106M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61400K .......... .......... .......... .......... .......... 66% 87.6M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61450K .......... .......... .......... .......... .......... 66% 111M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61500K .......... .......... .......... .......... .......... 66% 111M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61550K .......... .......... .......... .......... .......... 66% 114M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61600K .......... .......... .......... .......... .......... 66% 122M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61650K .......... .......... .......... .......... .......... 66% 89.2M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61700K .......... .......... .......... .......... .......... 66% 116M 0s 2024-05-09T20:29:13.341 INFO:teuthology.orchestra.run.smithi063.stderr: 61750K .......... .......... .......... .......... .......... 66% 112M 0s 2024-05-09T20:29:13.342 INFO:teuthology.orchestra.run.smithi063.stderr: 61800K .......... .......... .......... .......... .......... 66% 116M 0s 2024-05-09T20:29:13.342 INFO:teuthology.orchestra.run.smithi063.stderr: 61850K .......... .......... .......... .......... .......... 66% 110M 0s 2024-05-09T20:29:13.342 INFO:teuthology.orchestra.run.smithi063.stderr: 61900K .......... .......... .......... .......... .......... 66% 116M 0s 2024-05-09T20:29:13.342 INFO:teuthology.orchestra.run.smithi063.stderr: 61950K .......... .......... .......... .......... .......... 66% 112M 0s 2024-05-09T20:29:13.342 INFO:teuthology.orchestra.run.smithi063.stderr: 62000K .......... .......... .......... .......... .......... 66% 118M 0s 2024-05-09T20:29:13.342 INFO:teuthology.orchestra.run.smithi063.stderr: 62050K .......... .......... .......... .......... .......... 66% 89.5M 0s 2024-05-09T20:29:13.342 INFO:teuthology.orchestra.run.smithi063.stderr: 62100K .......... .......... .......... .......... .......... 66% 114M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62150K .......... .......... .......... .......... .......... 66% 115M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62200K .......... .......... .......... .......... .......... 67% 113M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62250K .......... .......... .......... .......... .......... 67% 109M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62300K .......... .......... .......... .......... .......... 67% 109M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62350K .......... .......... .......... .......... .......... 67% 117M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62400K .......... .......... .......... .......... .......... 67% 112M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62450K .......... .......... .......... .......... .......... 67% 90.6M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62500K .......... .......... .......... .......... .......... 67% 110M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62550K .......... .......... .......... .......... .......... 67% 124M 0s 2024-05-09T20:29:13.343 INFO:teuthology.orchestra.run.smithi063.stderr: 62600K .......... .......... .......... .......... .......... 67% 112M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 62650K .......... .......... .......... .......... .......... 67% 108M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 62700K .......... .......... .......... .......... .......... 67% 107M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 62750K .......... .......... .......... .......... .......... 67% 116M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 62800K .......... .......... .......... .......... .......... 67% 117M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 62850K .......... .......... .......... .......... .......... 67% 86.3M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 62900K .......... .......... .......... .......... .......... 67% 110M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 62950K .......... .......... .......... .......... .......... 67% 126M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 63000K .......... .......... .......... .......... .......... 67% 106M 0s 2024-05-09T20:29:13.344 INFO:teuthology.orchestra.run.smithi063.stderr: 63050K .......... .......... .......... .......... .......... 67% 107M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63100K .......... .......... .......... .......... .......... 67% 117M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63150K .......... .......... .......... .......... .......... 68% 118M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63200K .......... .......... .......... .......... .......... 68% 109M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63250K .......... .......... .......... .......... .......... 68% 86.8M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63300K .......... .......... .......... .......... .......... 68% 115M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63350K .......... .......... .......... .......... .......... 68% 48.3M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63400K .......... .......... .......... .......... .......... 68% 99.0M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63450K .......... .......... .......... .......... .......... 68% 106M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63500K .......... .......... .......... .......... .......... 68% 90.1M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63550K .......... .......... .......... .......... .......... 68% 122M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63600K .......... .......... .......... .......... .......... 68% 122M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63650K .......... .......... .......... .......... .......... 68% 99.9M 0s 2024-05-09T20:29:13.345 INFO:teuthology.orchestra.run.smithi063.stderr: 63700K .......... .......... .......... .......... .......... 68% 124M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 63750K .......... .......... .......... .......... .......... 68% 119M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 63800K .......... .......... .......... .......... .......... 68% 118M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 63850K .......... .......... .......... .......... .......... 68% 111M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 63900K .......... .......... .......... .......... .......... 68% 113M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 63950K .......... .......... .......... .......... .......... 68% 78.9M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 64000K .......... .......... .......... .......... .......... 68% 165M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 64050K .......... .......... .......... .......... .......... 68% 95.8M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 64100K .......... .......... .......... .......... .......... 69% 124M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 64150K .......... .......... .......... .......... .......... 69% 111M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 64200K .......... .......... .......... .......... .......... 69% 110M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 64250K .......... .......... .......... .......... .......... 69% 102M 0s 2024-05-09T20:29:13.346 INFO:teuthology.orchestra.run.smithi063.stderr: 64300K .......... .......... .......... .......... .......... 69% 107M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64350K .......... .......... .......... .......... .......... 69% 136M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64400K .......... .......... .......... .......... .......... 69% 110M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64450K .......... .......... .......... .......... .......... 69% 86.0M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64500K .......... .......... .......... .......... .......... 69% 117M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64550K .......... .......... .......... .......... .......... 69% 113M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64600K .......... .......... .......... .......... .......... 69% 118M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64650K .......... .......... .......... .......... .......... 69% 106M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64700K .......... .......... .......... .......... .......... 69% 119M 0s 2024-05-09T20:29:13.347 INFO:teuthology.orchestra.run.smithi063.stderr: 64750K .......... .......... .......... .......... .......... 69% 110M 0s 2024-05-09T20:29:13.348 INFO:teuthology.orchestra.run.smithi063.stderr: 64800K .......... .......... .......... .......... .......... 69% 112M 0s 2024-05-09T20:29:13.348 INFO:teuthology.orchestra.run.smithi063.stderr: 64850K .......... .......... .......... .......... .......... 69% 78.8M 0s 2024-05-09T20:29:13.348 INFO:teuthology.orchestra.run.smithi063.stderr: 64900K .......... .......... .......... .......... .......... 69% 41.5M 0s 2024-05-09T20:29:13.348 INFO:teuthology.orchestra.run.smithi063.stderr: 64950K .......... .......... .......... .......... .......... 69% 174M 0s 2024-05-09T20:29:13.348 INFO:teuthology.orchestra.run.smithi063.stderr: 65000K .......... .......... .......... .......... .......... 70% 109M 0s 2024-05-09T20:29:13.348 INFO:teuthology.orchestra.run.smithi063.stderr: 65050K .......... .......... .......... .......... .......... 70% 113M 0s 2024-05-09T20:29:13.348 INFO:teuthology.orchestra.run.smithi063.stderr: 65100K .......... .......... .......... .......... .......... 70% 106M 0s 2024-05-09T20:29:13.348 INFO:teuthology.orchestra.run.smithi063.stderr: 65150K .......... .......... .......... .......... .......... 70% 107M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65200K .......... .......... .......... .......... .......... 70% 130M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65250K .......... .......... .......... .......... .......... 70% 91.9M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65300K .......... .......... .......... .......... .......... 70% 114M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65350K .......... .......... .......... .......... .......... 70% 110M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65400K .......... .......... .......... .......... .......... 70% 103M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65450K .......... .......... .......... .......... .......... 70% 116M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65500K .......... .......... .......... .......... .......... 70% 106M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65550K .......... .......... .......... .......... .......... 70% 113M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65600K .......... .......... .......... .......... .......... 70% 94.0M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65650K .......... .......... .......... .......... .......... 70% 108M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65700K .......... .......... .......... .......... .......... 70% 114M 0s 2024-05-09T20:29:13.349 INFO:teuthology.orchestra.run.smithi063.stderr: 65750K .......... .......... .......... .......... .......... 70% 115M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 65800K .......... .......... .......... .......... .......... 70% 115M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 65850K .......... .......... .......... .......... .......... 70% 103M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 65900K .......... .......... .......... .......... .......... 70% 112M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 65950K .......... .......... .......... .......... .......... 71% 121M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66000K .......... .......... .......... .......... .......... 71% 118M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66050K .......... .......... .......... .......... .......... 71% 84.7M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66100K .......... .......... .......... .......... .......... 71% 113M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66150K .......... .......... .......... .......... .......... 71% 102M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66200K .......... .......... .......... .......... .......... 71% 125M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66250K .......... .......... .......... .......... .......... 71% 105M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66300K .......... .......... .......... .......... .......... 71% 108M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66350K .......... .......... .......... .......... .......... 71% 117M 0s 2024-05-09T20:29:13.350 INFO:teuthology.orchestra.run.smithi063.stderr: 66400K .......... .......... .......... .......... .......... 71% 122M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66450K .......... .......... .......... .......... .......... 71% 86.2M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66500K .......... .......... .......... .......... .......... 71% 110M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66550K .......... .......... .......... .......... .......... 71% 111M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66600K .......... .......... .......... .......... .......... 71% 111M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66650K .......... .......... .......... .......... .......... 71% 109M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66700K .......... .......... .......... .......... .......... 71% 104M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66750K .......... .......... .......... .......... .......... 71% 117M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66800K .......... .......... .......... .......... .......... 71% 106M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66850K .......... .......... .......... .......... .......... 72% 96.5M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66900K .......... .......... .......... .......... .......... 72% 115M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 66950K .......... .......... .......... .......... .......... 72% 117M 0s 2024-05-09T20:29:13.351 INFO:teuthology.orchestra.run.smithi063.stderr: 67000K .......... .......... .......... .......... .......... 72% 111M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67050K .......... .......... .......... .......... .......... 72% 95.0M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67100K .......... .......... .......... .......... .......... 72% 114M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67150K .......... .......... .......... .......... .......... 72% 120M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67200K .......... .......... .......... .......... .......... 72% 129M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67250K .......... .......... .......... .......... .......... 72% 88.3M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67300K .......... .......... .......... .......... .......... 72% 123M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67350K .......... .......... .......... .......... .......... 72% 116M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67400K .......... .......... .......... .......... .......... 72% 113M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67450K .......... .......... .......... .......... .......... 72% 103M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67500K .......... .......... .......... .......... .......... 72% 108M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67550K .......... .......... .......... .......... .......... 72% 83.5M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67600K .......... .......... .......... .......... .......... 72% 96.4M 0s 2024-05-09T20:29:13.352 INFO:teuthology.orchestra.run.smithi063.stderr: 67650K .......... .......... .......... .......... .......... 72% 133M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 67700K .......... .......... .......... .......... .......... 72% 123M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 67750K .......... .......... .......... .......... .......... 72% 112M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 67800K .......... .......... .......... .......... .......... 73% 113M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 67850K .......... .......... .......... .......... .......... 73% 121M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 67900K .......... .......... .......... .......... .......... 73% 104M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 67950K .......... .......... .......... .......... .......... 73% 93.9M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 68000K .......... .......... .......... .......... .......... 73% 110M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 68050K .......... .......... .......... .......... .......... 73% 117M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 68100K .......... .......... .......... .......... .......... 73% 116M 0s 2024-05-09T20:29:13.353 INFO:teuthology.orchestra.run.smithi063.stderr: 68150K .......... .......... .......... .......... .......... 73% 118M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68200K .......... .......... .......... .......... .......... 73% 113M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68250K .......... .......... .......... .......... .......... 73% 91.1M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68300K .......... .......... .......... .......... .......... 73% 116M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68350K .......... .......... .......... .......... .......... 73% 109M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68400K .......... .......... .......... .......... .......... 73% 119M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68450K .......... .......... .......... .......... .......... 73% 107M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68500K .......... .......... .......... .......... .......... 73% 114M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68550K .......... .......... .......... .......... .......... 73% 112M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68600K .......... .......... .......... .......... .......... 73% 119M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68650K .......... .......... .......... .......... .......... 73% 86.5M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68700K .......... .......... .......... .......... .......... 74% 115M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68750K .......... .......... .......... .......... .......... 74% 115M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68800K .......... .......... .......... .......... .......... 74% 105M 0s 2024-05-09T20:29:13.354 INFO:teuthology.orchestra.run.smithi063.stderr: 68850K .......... .......... .......... .......... .......... 74% 113M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 68900K .......... .......... .......... .......... .......... 74% 99.0M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 68950K .......... .......... .......... .......... .......... 74% 140M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69000K .......... .......... .......... .......... .......... 74% 119M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69050K .......... .......... .......... .......... .......... 74% 88.3M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69100K .......... .......... .......... .......... .......... 74% 122M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69150K .......... .......... .......... .......... .......... 74% 115M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69200K .......... .......... .......... .......... .......... 74% 112M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69250K .......... .......... .......... .......... .......... 74% 106M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69300K .......... .......... .......... .......... .......... 74% 114M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69350K .......... .......... .......... .......... .......... 74% 110M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69400K .......... .......... .......... .......... .......... 74% 122M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69450K .......... .......... .......... .......... .......... 74% 90.8M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69500K .......... .......... .......... .......... .......... 74% 112M 0s 2024-05-09T20:29:13.355 INFO:teuthology.orchestra.run.smithi063.stderr: 69550K .......... .......... .......... .......... .......... 74% 114M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 69600K .......... .......... .......... .......... .......... 74% 113M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 69650K .......... .......... .......... .......... .......... 75% 89.1M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 69700K .......... .......... .......... .......... .......... 75% 127M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 69750K .......... .......... .......... .......... .......... 75% 120M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 69800K .......... .......... .......... .......... .......... 75% 104M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 69850K .......... .......... .......... .......... .......... 75% 98.7M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 69900K .......... .......... .......... .......... .......... 75% 118M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 69950K .......... .......... .......... .......... .......... 75% 114M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 70000K .......... .......... .......... .......... .......... 75% 79.7M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 70050K .......... .......... .......... .......... .......... 75% 188M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 70100K .......... .......... .......... .......... .......... 75% 118M 0s 2024-05-09T20:29:13.356 INFO:teuthology.orchestra.run.smithi063.stderr: 70150K .......... .......... .......... .......... .......... 75% 103M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70200K .......... .......... .......... .......... .......... 75% 100M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70250K .......... .......... .......... .......... .......... 75% 46.1M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70300K .......... .......... .......... .......... .......... 75% 112M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70350K .......... .......... .......... .......... .......... 75% 98.8M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70400K .......... .......... .......... .......... .......... 75% 127M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70450K .......... .......... .......... .......... .......... 75% 124M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70500K .......... .......... .......... .......... .......... 75% 121M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70550K .......... .......... .......... .......... .......... 75% 91.1M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70600K .......... .......... .......... .......... .......... 76% 114M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70650K .......... .......... .......... .......... .......... 76% 110M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70700K .......... .......... .......... .......... .......... 76% 121M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70750K .......... .......... .......... .......... .......... 76% 104M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70800K .......... .......... .......... .......... .......... 76% 119M 0s 2024-05-09T20:29:13.357 INFO:teuthology.orchestra.run.smithi063.stderr: 70850K .......... .......... .......... .......... .......... 76% 112M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 70900K .......... .......... .......... .......... .......... 76% 113M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 70950K .......... .......... .......... .......... .......... 76% 90.2M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71000K .......... .......... .......... .......... .......... 76% 118M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71050K .......... .......... .......... .......... .......... 76% 117M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71100K .......... .......... .......... .......... .......... 76% 110M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71150K .......... .......... .......... .......... .......... 76% 110M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71200K .......... .......... .......... .......... .......... 76% 113M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71250K .......... .......... .......... .......... .......... 76% 113M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71300K .......... .......... .......... .......... .......... 76% 118M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71350K .......... .......... .......... .......... .......... 76% 83.0M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71400K .......... .......... .......... .......... .......... 76% 112M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71450K .......... .......... .......... .......... .......... 76% 114M 0s 2024-05-09T20:29:13.358 INFO:teuthology.orchestra.run.smithi063.stderr: 71500K .......... .......... .......... .......... .......... 77% 114M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71550K .......... .......... .......... .......... .......... 77% 109M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71600K .......... .......... .......... .......... .......... 77% 41.5M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71650K .......... .......... .......... .......... .......... 77% 30.7M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71700K .......... .......... .......... .......... .......... 77% 116M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71750K .......... .......... .......... .......... .......... 77% 76.1M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71800K .......... .......... .......... .......... .......... 77% 107M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71850K .......... .......... .......... .......... .......... 77% 103M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71900K .......... .......... .......... .......... .......... 77% 140M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 71950K .......... .......... .......... .......... .......... 77% 98.4M 0s 2024-05-09T20:29:13.359 INFO:teuthology.orchestra.run.smithi063.stderr: 72000K .......... .......... .......... .......... .......... 77% 113M 0s 2024-05-09T20:29:13.360 INFO:teuthology.orchestra.run.smithi063.stderr: 72050K .......... .......... .......... .......... .......... 77% 116M 0s 2024-05-09T20:29:13.360 INFO:teuthology.orchestra.run.smithi063.stderr: 72100K .......... .......... .......... .......... .......... 77% 108M 0s 2024-05-09T20:29:13.360 INFO:teuthology.orchestra.run.smithi063.stderr: 72150K .......... .......... .......... .......... .......... 77% 94.1M 0s 2024-05-09T20:29:13.360 INFO:teuthology.orchestra.run.smithi063.stderr: 72200K .......... .......... .......... .......... .......... 77% 115M 0s 2024-05-09T20:29:13.360 INFO:teuthology.orchestra.run.smithi063.stderr: 72250K .......... .......... .......... .......... .......... 77% 114M 0s 2024-05-09T20:29:13.360 INFO:teuthology.orchestra.run.smithi063.stderr: 72300K .......... .......... .......... .......... .......... 77% 114M 0s 2024-05-09T20:29:13.360 INFO:teuthology.orchestra.run.smithi063.stderr: 72350K .......... .......... .......... .......... .......... 77% 107M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72400K .......... .......... .......... .......... .......... 77% 122M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72450K .......... .......... .......... .......... .......... 78% 112M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72500K .......... .......... .......... .......... .......... 78% 115M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72550K .......... .......... .......... .......... .......... 78% 91.6M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72600K .......... .......... .......... .......... .......... 78% 117M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72650K .......... .......... .......... .......... .......... 78% 119M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72700K .......... .......... .......... .......... .......... 78% 109M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72750K .......... .......... .......... .......... .......... 78% 115M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72800K .......... .......... .......... .......... .......... 78% 120M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72850K .......... .......... .......... .......... .......... 78% 84.2M 0s 2024-05-09T20:29:13.361 INFO:teuthology.orchestra.run.smithi063.stderr: 72900K .......... .......... .......... .......... .......... 78% 120M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 72950K .......... .......... .......... .......... .......... 78% 124M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73000K .......... .......... .......... .......... .......... 78% 115M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73050K .......... .......... .......... .......... .......... 78% 105M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73100K .......... .......... .......... .......... .......... 78% 113M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73150K .......... .......... .......... .......... .......... 78% 119M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73200K .......... .......... .......... .......... .......... 78% 114M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73250K .......... .......... .......... .......... .......... 78% 84.9M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73300K .......... .......... .......... .......... .......... 78% 109M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73350K .......... .......... .......... .......... .......... 79% 113M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73400K .......... .......... .......... .......... .......... 79% 112M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73450K .......... .......... .......... .......... .......... 79% 100M 0s 2024-05-09T20:29:13.362 INFO:teuthology.orchestra.run.smithi063.stderr: 73500K .......... .......... .......... .......... .......... 79% 135M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73550K .......... .......... .......... .......... .......... 79% 116M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73600K .......... .......... .......... .......... .......... 79% 67.0M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73650K .......... .......... .......... .......... .......... 79% 57.1M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73700K .......... .......... .......... .......... .......... 79% 108M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73750K .......... .......... .......... .......... .......... 79% 121M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73800K .......... .......... .......... .......... .......... 79% 115M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73850K .......... .......... .......... .......... .......... 79% 104M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73900K .......... .......... .......... .......... .......... 79% 119M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 73950K .......... .......... .......... .......... .......... 79% 111M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 74000K .......... .......... .......... .......... .......... 79% 95.5M 0s 2024-05-09T20:29:13.363 INFO:teuthology.orchestra.run.smithi063.stderr: 74050K .......... .......... .......... .......... .......... 79% 95.6M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74100K .......... .......... .......... .......... .......... 79% 103M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74150K .......... .......... .......... .......... .......... 79% 112M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74200K .......... .......... .......... .......... .......... 79% 117M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74250K .......... .......... .......... .......... .......... 79% 101M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74300K .......... .......... .......... .......... .......... 80% 123M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74350K .......... .......... .......... .......... .......... 80% 117M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74400K .......... .......... .......... .......... .......... 80% 117M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74450K .......... .......... .......... .......... .......... 80% 88.2M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74500K .......... .......... .......... .......... .......... 80% 119M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74550K .......... .......... .......... .......... .......... 80% 99.0M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74600K .......... .......... .......... .......... .......... 80% 116M 0s 2024-05-09T20:29:13.364 INFO:teuthology.orchestra.run.smithi063.stderr: 74650K .......... .......... .......... .......... .......... 80% 90.9M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 74700K .......... .......... .......... .......... .......... 80% 116M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 74750K .......... .......... .......... .......... .......... 80% 94.1M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 74800K .......... .......... .......... .......... .......... 80% 124M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 74850K .......... .......... .......... .......... .......... 80% 111M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 74900K .......... .......... .......... .......... .......... 80% 83.9M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 74950K .......... .......... .......... .......... .......... 80% 111M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 75000K .......... .......... .......... .......... .......... 80% 112M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 75050K .......... .......... .......... .......... .......... 80% 120M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 75100K .......... .......... .......... .......... .......... 80% 102M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 75150K .......... .......... .......... .......... .......... 80% 38.8M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 75200K .......... .......... .......... .......... .......... 81% 143M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 75250K .......... .......... .......... .......... .......... 81% 129M 0s 2024-05-09T20:29:13.365 INFO:teuthology.orchestra.run.smithi063.stderr: 75300K .......... .......... .......... .......... .......... 81% 108M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75350K .......... .......... .......... .......... .......... 81% 113M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75400K .......... .......... .......... .......... .......... 81% 115M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75450K .......... .......... .......... .......... .......... 81% 117M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75500K .......... .......... .......... .......... .......... 81% 108M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75550K .......... .......... .......... .......... .......... 81% 94.1M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75600K .......... .......... .......... .......... .......... 81% 118M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75650K .......... .......... .......... .......... .......... 81% 120M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75700K .......... .......... .......... .......... .......... 81% 108M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75750K .......... .......... .......... .......... .......... 81% 110M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75800K .......... .......... .......... .......... .......... 81% 111M 0s 2024-05-09T20:29:13.366 INFO:teuthology.orchestra.run.smithi063.stderr: 75850K .......... .......... .......... .......... .......... 81% 119M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 75900K .......... .......... .......... .......... .......... 81% 114M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 75950K .......... .......... .......... .......... .......... 81% 92.8M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 76000K .......... .......... .......... .......... .......... 81% 119M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 76050K .......... .......... .......... .......... .......... 81% 110M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 76100K .......... .......... .......... .......... .......... 81% 117M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 76150K .......... .......... .......... .......... .......... 82% 101M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 76200K .......... .......... .......... .......... .......... 82% 132M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 76250K .......... .......... .......... .......... .......... 82% 105M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 76300K .......... .......... .......... .......... .......... 82% 114M 0s 2024-05-09T20:29:13.367 INFO:teuthology.orchestra.run.smithi063.stderr: 76350K .......... .......... .......... .......... .......... 82% 93.6M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76400K .......... .......... .......... .......... .......... 82% 116M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76450K .......... .......... .......... .......... .......... 82% 113M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76500K .......... .......... .......... .......... .......... 82% 113M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76550K .......... .......... .......... .......... .......... 82% 104M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76600K .......... .......... .......... .......... .......... 82% 104M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76650K .......... .......... .......... .......... .......... 82% 129M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76700K .......... .......... .......... .......... .......... 82% 103M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76750K .......... .......... .......... .......... .......... 82% 104M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76800K .......... .......... .......... .......... .......... 82% 124M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76850K .......... .......... .......... .......... .......... 82% 107M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76900K .......... .......... .......... .......... .......... 82% 109M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 76950K .......... .......... .......... .......... .......... 82% 102M 0s 2024-05-09T20:29:13.368 INFO:teuthology.orchestra.run.smithi063.stderr: 77000K .......... .......... .......... .......... .......... 82% 113M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77050K .......... .......... .......... .......... .......... 82% 59.7M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77100K .......... .......... .......... .......... .......... 83% 77.3M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77150K .......... .......... .......... .......... .......... 83% 65.6M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77200K .......... .......... .......... .......... .......... 83% 75.8M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77250K .......... .......... .......... .......... .......... 83% 76.0M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77300K .......... .......... .......... .......... .......... 83% 76.6M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77350K .......... .......... .......... .......... .......... 83% 70.6M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77400K .......... .......... .......... .......... .......... 83% 77.8M 0s 2024-05-09T20:29:13.369 INFO:teuthology.orchestra.run.smithi063.stderr: 77450K .......... .......... .......... .......... .......... 83% 79.6M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77500K .......... .......... .......... .......... .......... 83% 61.5M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77550K .......... .......... .......... .......... .......... 83% 80.6M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77600K .......... .......... .......... .......... .......... 83% 80.6M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77650K .......... .......... .......... .......... .......... 83% 89.4M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77700K .......... .......... .......... .......... .......... 83% 88.4M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77750K .......... .......... .......... .......... .......... 83% 83.5M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77800K .......... .......... .......... .......... .......... 83% 70.9M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77850K .......... .......... .......... .......... .......... 83% 83.0M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77900K .......... .......... .......... .......... .......... 83% 89.2M 0s 2024-05-09T20:29:13.370 INFO:teuthology.orchestra.run.smithi063.stderr: 77950K .......... .......... .......... .......... .......... 83% 77.7M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78000K .......... .......... .......... .......... .......... 84% 82.7M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78050K .......... .......... .......... .......... .......... 84% 83.8M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78100K .......... .......... .......... .......... .......... 84% 86.9M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78150K .......... .......... .......... .......... .......... 84% 82.2M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78200K .......... .......... .......... .......... .......... 84% 84.5M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78250K .......... .......... .......... .......... .......... 84% 73.1M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78300K .......... .......... .......... .......... .......... 84% 89.8M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78350K .......... .......... .......... .......... .......... 84% 94.3M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78400K .......... .......... .......... .......... .......... 84% 92.5M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78450K .......... .......... .......... .......... .......... 84% 93.4M 0s 2024-05-09T20:29:13.371 INFO:teuthology.orchestra.run.smithi063.stderr: 78500K .......... .......... .......... .......... .......... 84% 80.0M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78550K .......... .......... .......... .......... .......... 84% 89.5M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78600K .......... .......... .......... .......... .......... 84% 98.9M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78650K .......... .......... .......... .......... .......... 84% 81.3M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78700K .......... .......... .......... .......... .......... 84% 91.8M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78750K .......... .......... .......... .......... .......... 84% 92.8M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78800K .......... .......... .......... .......... .......... 84% 95.1M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78850K .......... .......... .......... .......... .......... 84% 103M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78900K .......... .......... .......... .......... .......... 84% 101M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 78950K .......... .......... .......... .......... .......... 85% 76.8M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 79000K .......... .......... .......... .......... .......... 85% 103M 0s 2024-05-09T20:29:13.372 INFO:teuthology.orchestra.run.smithi063.stderr: 79050K .......... .......... .......... .......... .......... 85% 107M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79100K .......... .......... .......... .......... .......... 85% 108M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79150K .......... .......... .......... .......... .......... 85% 180M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79200K .......... .......... .......... .......... .......... 85% 156M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79250K .......... .......... .......... .......... .......... 85% 189M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79300K .......... .......... .......... .......... .......... 85% 199M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79350K .......... .......... .......... .......... .......... 85% 199M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79400K .......... .......... .......... .......... .......... 85% 164M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79450K .......... .......... .......... .......... .......... 85% 194M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79500K .......... .......... .......... .......... .......... 85% 157M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79550K .......... .......... .......... .......... .......... 85% 199M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79600K .......... .......... .......... .......... .......... 85% 198M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79650K .......... .......... .......... .......... .......... 85% 170M 0s 2024-05-09T20:29:13.373 INFO:teuthology.orchestra.run.smithi063.stderr: 79700K .......... .......... .......... .......... .......... 85% 199M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 79750K .......... .......... .......... .......... .......... 85% 201M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 79800K .......... .......... .......... .......... .......... 85% 202M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 79850K .......... .......... .......... .......... .......... 86% 168M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 79900K .......... .......... .......... .......... .......... 86% 198M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 79950K .......... .......... .......... .......... .......... 86% 194M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 80000K .......... .......... .......... .......... .......... 86% 300M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 80050K .......... .......... .......... .......... .......... 86% 286M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 80100K .......... .......... .......... .......... .......... 86% 330M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 80150K .......... .......... .......... .......... .......... 86% 341M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 80200K .......... .......... .......... .......... .......... 86% 186M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 80250K .......... .......... .......... .......... .......... 86% 171M 0s 2024-05-09T20:29:13.374 INFO:teuthology.orchestra.run.smithi063.stderr: 80300K .......... .......... .......... .......... .......... 86% 194M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80350K .......... .......... .......... .......... .......... 86% 193M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80400K .......... .......... .......... .......... .......... 86% 197M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80450K .......... .......... .......... .......... .......... 86% 149M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80500K .......... .......... .......... .......... .......... 86% 121M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80550K .......... .......... .......... .......... .......... 86% 104M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80600K .......... .......... .......... .......... .......... 86% 109M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80650K .......... .......... .......... .......... .......... 86% 92.9M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80700K .......... .......... .......... .......... .......... 86% 119M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80750K .......... .......... .......... .......... .......... 86% 109M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80800K .......... .......... .......... .......... .......... 87% 114M 0s 2024-05-09T20:29:13.375 INFO:teuthology.orchestra.run.smithi063.stderr: 80850K .......... .......... .......... .......... .......... 87% 107M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 80900K .......... .......... .......... .......... .......... 87% 116M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 80950K .......... .......... .......... .......... .......... 87% 103M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81000K .......... .......... .......... .......... .......... 87% 118M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81050K .......... .......... .......... .......... .......... 87% 89.3M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81100K .......... .......... .......... .......... .......... 87% 116M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81150K .......... .......... .......... .......... .......... 87% 111M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81200K .......... .......... .......... .......... .......... 87% 119M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81250K .......... .......... .......... .......... .......... 87% 105M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81300K .......... .......... .......... .......... .......... 87% 117M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81350K .......... .......... .......... .......... .......... 87% 117M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81400K .......... .......... .......... .......... .......... 87% 118M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81450K .......... .......... .......... .......... .......... 87% 88.9M 0s 2024-05-09T20:29:13.376 INFO:teuthology.orchestra.run.smithi063.stderr: 81500K .......... .......... .......... .......... .......... 87% 110M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81550K .......... .......... .......... .......... .......... 87% 117M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81600K .......... .......... .......... .......... .......... 87% 112M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81650K .......... .......... .......... .......... .......... 87% 110M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81700K .......... .......... .......... .......... .......... 87% 121M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81750K .......... .......... .......... .......... .......... 88% 103M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81800K .......... .......... .......... .......... .......... 88% 122M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81850K .......... .......... .......... .......... .......... 88% 93.1M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81900K .......... .......... .......... .......... .......... 88% 115M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 81950K .......... .......... .......... .......... .......... 88% 102M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 82000K .......... .......... .......... .......... .......... 88% 112M 0s 2024-05-09T20:29:13.377 INFO:teuthology.orchestra.run.smithi063.stderr: 82050K .......... .......... .......... .......... .......... 88% 121M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82100K .......... .......... .......... .......... .......... 88% 112M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82150K .......... .......... .......... .......... .......... 88% 96.2M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82200K .......... .......... .......... .......... .......... 88% 111M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82250K .......... .......... .......... .......... .......... 88% 115M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82300K .......... .......... .......... .......... .......... 88% 112M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82350K .......... .......... .......... .......... .......... 88% 114M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82400K .......... .......... .......... .......... .......... 88% 114M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82450K .......... .......... .......... .......... .......... 88% 114M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82500K .......... .......... .......... .......... .......... 88% 117M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82550K .......... .......... .......... .......... .......... 88% 90.8M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82600K .......... .......... .......... .......... .......... 88% 112M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82650K .......... .......... .......... .......... .......... 89% 110M 0s 2024-05-09T20:29:13.378 INFO:teuthology.orchestra.run.smithi063.stderr: 82700K .......... .......... .......... .......... .......... 89% 114M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 82750K .......... .......... .......... .......... .......... 89% 107M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 82800K .......... .......... .......... .......... .......... 89% 113M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 82850K .......... .......... .......... .......... .......... 89% 115M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 82900K .......... .......... .......... .......... .......... 89% 113M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 82950K .......... .......... .......... .......... .......... 89% 102M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 83000K .......... .......... .......... .......... .......... 89% 118M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 83050K .......... .......... .......... .......... .......... 89% 116M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 83100K .......... .......... .......... .......... .......... 89% 116M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 83150K .......... .......... .......... .......... .......... 89% 103M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 83200K .......... .......... .......... .......... .......... 89% 116M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 83250K .......... .......... .......... .......... .......... 89% 115M 0s 2024-05-09T20:29:13.379 INFO:teuthology.orchestra.run.smithi063.stderr: 83300K .......... .......... .......... .......... .......... 89% 114M 0s 2024-05-09T20:29:13.380 INFO:teuthology.orchestra.run.smithi063.stderr: 83350K .......... .......... .......... .......... .......... 89% 92.7M 0s 2024-05-09T20:29:13.380 INFO:teuthology.orchestra.run.smithi063.stderr: 83400K .......... .......... .......... .......... .......... 89% 116M 0s 2024-05-09T20:29:13.380 INFO:teuthology.orchestra.run.smithi063.stderr: 83450K .......... .......... .......... .......... .......... 89% 114M 0s 2024-05-09T20:29:13.380 INFO:teuthology.orchestra.run.smithi063.stderr: 83500K .......... .......... .......... .......... .......... 89% 119M 0s 2024-05-09T20:29:13.380 INFO:teuthology.orchestra.run.smithi063.stderr: 83550K .......... .......... .......... .......... .......... 89% 107M 0s 2024-05-09T20:29:13.380 INFO:teuthology.orchestra.run.smithi063.stderr: 83600K .......... .......... .......... .......... .......... 90% 112M 0s 2024-05-09T20:29:13.380 INFO:teuthology.orchestra.run.smithi063.stderr: 83650K .......... .......... .......... .......... .......... 90% 115M 0s 2024-05-09T20:29:13.380 INFO:teuthology.orchestra.run.smithi063.stderr: 83700K .......... .......... .......... .......... .......... 90% 117M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 83750K .......... .......... .......... .......... .......... 90% 91.6M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 83800K .......... .......... .......... .......... .......... 90% 99.8M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 83850K .......... .......... .......... .......... .......... 90% 113M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 83900K .......... .......... .......... .......... .......... 90% 45.6M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 83950K .......... .......... .......... .......... .......... 90% 104M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 84000K .......... .......... .......... .......... .......... 90% 118M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 84050K .......... .......... .......... .......... .......... 90% 110M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 84100K .......... .......... .......... .......... .......... 90% 114M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 84150K .......... .......... .......... .......... .......... 90% 101M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 84200K .......... .......... .......... .......... .......... 90% 112M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 84250K .......... .......... .......... .......... .......... 90% 109M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 84300K .......... .......... .......... .......... .......... 90% 114M 0s 2024-05-09T20:29:13.381 INFO:teuthology.orchestra.run.smithi063.stderr: 84350K .......... .......... .......... .......... .......... 90% 106M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84400K .......... .......... .......... .......... .......... 90% 125M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84450K .......... .......... .......... .......... .......... 90% 88.9M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84500K .......... .......... .......... .......... .......... 91% 111M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84550K .......... .......... .......... .......... .......... 91% 114M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84600K .......... .......... .......... .......... .......... 91% 115M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84650K .......... .......... .......... .......... .......... 91% 106M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84700K .......... .......... .......... .......... .......... 91% 115M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84750K .......... .......... .......... .......... .......... 91% 116M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84800K .......... .......... .......... .......... .......... 91% 116M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84850K .......... .......... .......... .......... .......... 91% 96.8M 0s 2024-05-09T20:29:13.382 INFO:teuthology.orchestra.run.smithi063.stderr: 84900K .......... .......... .......... .......... .......... 91% 112M 0s 2024-05-09T20:29:13.383 INFO:teuthology.orchestra.run.smithi063.stderr: 84950K .......... .......... .......... .......... .......... 91% 114M 0s 2024-05-09T20:29:13.383 INFO:teuthology.orchestra.run.smithi063.stderr: 85000K .......... .......... .......... .......... .......... 91% 113M 0s 2024-05-09T20:29:13.383 INFO:teuthology.orchestra.run.smithi063.stderr: 85050K .......... .......... .......... .......... .......... 91% 106M 0s 2024-05-09T20:29:13.383 INFO:teuthology.orchestra.run.smithi063.stderr: 85100K .......... .......... .......... .......... .......... 91% 113M 0s 2024-05-09T20:29:13.383 INFO:teuthology.orchestra.run.smithi063.stderr: 85150K .......... .......... .......... .......... .......... 91% 119M 0s 2024-05-09T20:29:13.383 INFO:teuthology.orchestra.run.smithi063.stderr: 85200K .......... .......... .......... .......... .......... 91% 112M 0s 2024-05-09T20:29:13.383 INFO:teuthology.orchestra.run.smithi063.stderr: 85250K .......... .......... .......... .......... .......... 91% 89.3M 0s 2024-05-09T20:29:13.383 INFO:teuthology.orchestra.run.smithi063.stderr: 85300K .......... .......... .......... .......... .......... 91% 123M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85350K .......... .......... .......... .......... .......... 91% 65.8M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85400K .......... .......... .......... .......... .......... 91% 118M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85450K .......... .......... .......... .......... .......... 92% 109M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85500K .......... .......... .......... .......... .......... 92% 121M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85550K .......... .......... .......... .......... .......... 92% 123M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85600K .......... .......... .......... .......... .......... 92% 131M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85650K .......... .......... .......... .......... .......... 92% 50.3M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85700K .......... .......... .......... .......... .......... 92% 122M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85750K .......... .......... .......... .......... .......... 92% 116M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85800K .......... .......... .......... .......... .......... 92% 112M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85850K .......... .......... .......... .......... .......... 92% 111M 0s 2024-05-09T20:29:13.384 INFO:teuthology.orchestra.run.smithi063.stderr: 85900K .......... .......... .......... .......... .......... 92% 110M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 85950K .......... .......... .......... .......... .......... 92% 118M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 86000K .......... .......... .......... .......... .......... 92% 119M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 86050K .......... .......... .......... .......... .......... 92% 91.5M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 86100K .......... .......... .......... .......... .......... 92% 103M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 86150K .......... .......... .......... .......... .......... 92% 122M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 86200K .......... .......... .......... .......... .......... 92% 124M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 86250K .......... .......... .......... .......... .......... 92% 112M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 86300K .......... .......... .......... .......... .......... 92% 113M 0s 2024-05-09T20:29:13.385 INFO:teuthology.orchestra.run.smithi063.stderr: 86350K .......... .......... .......... .......... .......... 93% 116M 0s 2024-05-09T20:29:13.386 INFO:teuthology.orchestra.run.smithi063.stderr: 86400K .......... .......... .......... .......... .......... 93% 114M 0s 2024-05-09T20:29:13.386 INFO:teuthology.orchestra.run.smithi063.stderr: 86450K .......... .......... .......... .......... .......... 93% 94.8M 0s 2024-05-09T20:29:13.386 INFO:teuthology.orchestra.run.smithi063.stderr: 86500K .......... .......... .......... .......... .......... 93% 123M 0s 2024-05-09T20:29:13.386 INFO:teuthology.orchestra.run.smithi063.stderr: 86550K .......... .......... .......... .......... .......... 93% 98.6M 0s 2024-05-09T20:29:13.386 INFO:teuthology.orchestra.run.smithi063.stderr: 86600K .......... .......... .......... .......... .......... 93% 122M 0s 2024-05-09T20:29:13.386 INFO:teuthology.orchestra.run.smithi063.stderr: 86650K .......... .......... .......... .......... .......... 93% 109M 0s 2024-05-09T20:29:13.386 INFO:teuthology.orchestra.run.smithi063.stderr: 86700K .......... .......... .......... .......... .......... 93% 118M 0s 2024-05-09T20:29:13.387 INFO:teuthology.orchestra.run.smithi063.stderr: 86750K .......... .......... .......... .......... .......... 93% 89.0M 0s 2024-05-09T20:29:13.387 INFO:teuthology.orchestra.run.smithi063.stderr: 86800K .......... .......... .......... .......... .......... 93% 115M 0s 2024-05-09T20:29:13.387 INFO:teuthology.orchestra.run.smithi063.stderr: 86850K .......... .......... .......... .......... .......... 93% 118M 0s 2024-05-09T20:29:13.387 INFO:teuthology.orchestra.run.smithi063.stderr: 86900K .......... .......... .......... .......... .......... 93% 114M 0s 2024-05-09T20:29:13.387 INFO:teuthology.orchestra.run.smithi063.stderr: 86950K .......... .......... .......... .......... .......... 93% 110M 0s 2024-05-09T20:29:13.387 INFO:teuthology.orchestra.run.smithi063.stderr: 87000K .......... .......... .......... .......... .......... 93% 105M 0s 2024-05-09T20:29:13.387 INFO:teuthology.orchestra.run.smithi063.stderr: 87050K .......... .......... .......... .......... .......... 93% 122M 0s 2024-05-09T20:29:13.387 INFO:teuthology.orchestra.run.smithi063.stderr: 87100K .......... .......... .......... .......... .......... 93% 118M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87150K .......... .......... .......... .......... .......... 93% 91.6M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87200K .......... .......... .......... .......... .......... 93% 113M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87250K .......... .......... .......... .......... .......... 93% 116M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87300K .......... .......... .......... .......... .......... 94% 120M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87350K .......... .......... .......... .......... .......... 94% 106M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87400K .......... .......... .......... .......... .......... 94% 115M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87450K .......... .......... .......... .......... .......... 94% 113M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87500K .......... .......... .......... .......... .......... 94% 112M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87550K .......... .......... .......... .......... .......... 94% 91.3M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87600K .......... .......... .......... .......... .......... 94% 115M 0s 2024-05-09T20:29:13.388 INFO:teuthology.orchestra.run.smithi063.stderr: 87650K .......... .......... .......... .......... .......... 94% 120M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 87700K .......... .......... .......... .......... .......... 94% 112M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 87750K .......... .......... .......... .......... .......... 94% 103M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 87800K .......... .......... .......... .......... .......... 94% 112M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 87850K .......... .......... .......... .......... .......... 94% 118M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 87900K .......... .......... .......... .......... .......... 94% 113M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 87950K .......... .......... .......... .......... .......... 94% 92.1M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 88000K .......... .......... .......... .......... .......... 94% 112M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 88050K .......... .......... .......... .......... .......... 94% 115M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 88100K .......... .......... .......... .......... .......... 94% 114M 0s 2024-05-09T20:29:13.389 INFO:teuthology.orchestra.run.smithi063.stderr: 88150K .......... .......... .......... .......... .......... 94% 104M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88200K .......... .......... .......... .......... .......... 94% 116M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88250K .......... .......... .......... .......... .......... 95% 118M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88300K .......... .......... .......... .......... .......... 95% 113M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88350K .......... .......... .......... .......... .......... 95% 88.9M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88400K .......... .......... .......... .......... .......... 95% 119M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88450K .......... .......... .......... .......... .......... 95% 120M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88500K .......... .......... .......... .......... .......... 95% 107M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88550K .......... .......... .......... .......... .......... 95% 107M 0s 2024-05-09T20:29:13.390 INFO:teuthology.orchestra.run.smithi063.stderr: 88600K .......... .......... .......... .......... .......... 95% 113M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 88650K .......... .......... .......... .......... .......... 95% 116M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 88700K .......... .......... .......... .......... .......... 95% 116M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 88750K .......... .......... .......... .......... .......... 95% 87.8M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 88800K .......... .......... .......... .......... .......... 95% 119M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 88850K .......... .......... .......... .......... .......... 95% 113M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 88900K .......... .......... .......... .......... .......... 95% 96.7M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 88950K .......... .......... .......... .......... .......... 95% 100M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 89000K .......... .......... .......... .......... .......... 95% 114M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 89050K .......... .......... .......... .......... .......... 95% 114M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 89100K .......... .......... .......... .......... .......... 95% 90.4M 0s 2024-05-09T20:29:13.391 INFO:teuthology.orchestra.run.smithi063.stderr: 89150K .......... .......... .......... .......... .......... 96% 115M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89200K .......... .......... .......... .......... .......... 96% 99.0M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89250K .......... .......... .......... .......... .......... 96% 121M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89300K .......... .......... .......... .......... .......... 96% 107M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89350K .......... .......... .......... .......... .......... 96% 112M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89400K .......... .......... .......... .......... .......... 96% 117M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89450K .......... .......... .......... .......... .......... 96% 114M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89500K .......... .......... .......... .......... .......... 96% 89.1M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89550K .......... .......... .......... .......... .......... 96% 117M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89600K .......... .......... .......... .......... .......... 96% 106M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89650K .......... .......... .......... .......... .......... 96% 117M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89700K .......... .......... .......... .......... .......... 96% 105M 0s 2024-05-09T20:29:13.392 INFO:teuthology.orchestra.run.smithi063.stderr: 89750K .......... .......... .......... .......... .......... 96% 118M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 89800K .......... .......... .......... .......... .......... 96% 111M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 89850K .......... .......... .......... .......... .......... 96% 112M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 89900K .......... .......... .......... .......... .......... 96% 93.0M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 89950K .......... .......... .......... .......... .......... 96% 113M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 90000K .......... .......... .......... .......... .......... 96% 113M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 90050K .......... .......... .......... .......... .......... 96% 114M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 90100K .......... .......... .......... .......... .......... 97% 104M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 90150K .......... .......... .......... .......... .......... 97% 114M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 90200K .......... .......... .......... .......... .......... 97% 107M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 90250K .......... .......... .......... .......... .......... 97% 113M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 90300K .......... .......... .......... .......... .......... 97% 90.6M 0s 2024-05-09T20:29:13.393 INFO:teuthology.orchestra.run.smithi063.stderr: 90350K .......... .......... .......... .......... .......... 97% 114M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90400K .......... .......... .......... .......... .......... 97% 118M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90450K .......... .......... .......... .......... .......... 97% 117M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90500K .......... .......... .......... .......... .......... 97% 103M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90550K .......... .......... .......... .......... .......... 97% 109M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90600K .......... .......... .......... .......... .......... 97% 119M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90650K .......... .......... .......... .......... .......... 97% 115M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90700K .......... .......... .......... .......... .......... 97% 41.9M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90750K .......... .......... .......... .......... .......... 97% 120M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90800K .......... .......... .......... .......... .......... 97% 114M 0s 2024-05-09T20:29:13.394 INFO:teuthology.orchestra.run.smithi063.stderr: 90850K .......... .......... .......... .......... .......... 97% 115M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 90900K .......... .......... .......... .......... .......... 97% 108M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 90950K .......... .......... .......... .......... .......... 97% 118M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91000K .......... .......... .......... .......... .......... 98% 103M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91050K .......... .......... .......... .......... .......... 98% 114M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91100K .......... .......... .......... .......... .......... 98% 88.6M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91150K .......... .......... .......... .......... .......... 98% 113M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91200K .......... .......... .......... .......... .......... 98% 114M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91250K .......... .......... .......... .......... .......... 98% 113M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91300K .......... .......... .......... .......... .......... 98% 110M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91350K .......... .......... .......... .......... .......... 98% 79.1M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91400K .......... .......... .......... .......... .......... 98% 118M 0s 2024-05-09T20:29:13.395 INFO:teuthology.orchestra.run.smithi063.stderr: 91450K .......... .......... .......... .......... .......... 98% 103M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91500K .......... .......... .......... .......... .......... 98% 91.2M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91550K .......... .......... .......... .......... .......... 98% 116M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91600K .......... .......... .......... .......... .......... 98% 111M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91650K .......... .......... .......... .......... .......... 98% 115M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91700K .......... .......... .......... .......... .......... 98% 109M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91750K .......... .......... .......... .......... .......... 98% 114M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91800K .......... .......... .......... .......... .......... 98% 112M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91850K .......... .......... .......... .......... .......... 98% 116M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91900K .......... .......... .......... .......... .......... 98% 92.2M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 91950K .......... .......... .......... .......... .......... 99% 110M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 92000K .......... .......... .......... .......... .......... 99% 123M 0s 2024-05-09T20:29:13.396 INFO:teuthology.orchestra.run.smithi063.stderr: 92050K .......... .......... .......... .......... .......... 99% 111M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92100K .......... .......... .......... .......... .......... 99% 109M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92150K .......... .......... .......... .......... .......... 99% 110M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92200K .......... .......... .......... .......... .......... 99% 121M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92250K .......... .......... .......... .......... .......... 99% 113M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92300K .......... .......... .......... .......... .......... 99% 88.1M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92350K .......... .......... .......... .......... .......... 99% 122M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92400K .......... .......... .......... .......... .......... 99% 115M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92450K .......... .......... .......... .......... .......... 99% 113M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92500K .......... .......... .......... .......... .......... 99% 106M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92550K .......... .......... .......... .......... .......... 99% 113M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92600K .......... .......... .......... .......... .......... 99% 95.5M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92650K .......... .......... .......... .......... .......... 99% 116M 0s 2024-05-09T20:29:13.397 INFO:teuthology.orchestra.run.smithi063.stderr: 92700K .......... .......... .......... .......... .......... 99% 122M 0s 2024-05-09T20:29:13.398 INFO:teuthology.orchestra.run.smithi063.stderr: 92750K .......... .......... .......... .......... .......... 99% 113M 0s 2024-05-09T20:29:13.398 INFO:teuthology.orchestra.run.smithi063.stderr: 92800K .......... .......... .......... .......... .......... 99% 109M 0s 2024-05-09T20:29:13.398 INFO:teuthology.orchestra.run.smithi063.stderr: 92850K .......... .......... .......... .......... .......... 99% 91.7M 0s 2024-05-09T20:29:13.398 INFO:teuthology.orchestra.run.smithi063.stderr: 92900K 100% 134G=0.9s 2024-05-09T20:29:13.398 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-09T20:29:13.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09 20:29:12 (98.4 MB/s) - 'gradle-6.0.1-bin.zip' saved [95129672/95129672] 2024-05-09T20:29:13.398 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-05-09T20:29:13.431 INFO:teuthology.orchestra.run.smithi063.stdout:Archive: gradle-6.0.1-bin.zip 2024-05-09T20:29:13.431 INFO:teuthology.orchestra.run.smithi063.stdout: creating: /opt/gradle/gradle-6.0.1/ 2024-05-09T20:29:13.431 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/getting-started.html 2024-05-09T20:29:13.431 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/LICENSE 2024-05-09T20:29:13.431 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/NOTICE 2024-05-09T20:29:13.431 INFO:teuthology.orchestra.run.smithi063.stdout: creating: /opt/gradle/gradle-6.0.1/init.d/ 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/init.d/readme.txt 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: creating: /opt/gradle/gradle-6.0.1/bin/ 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/bin/gradle.bat 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: creating: /opt/gradle/gradle-6.0.1/lib/ 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-api-metadata-6.0.1.jar 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-launcher-6.0.1.jar 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-runtime-api-info-6.0.1.jar 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-6.0.1.jar 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-tooling-api-6.0.1.jar 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-wrapper-6.0.1.jar 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-installation-beacon-6.0.1.jar 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-bootstrap-6.0.1.jar 2024-05-09T20:29:13.432 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-file-collections-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-groovy-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-execution-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-model-core-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-packaging-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-cache-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-core-api-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-persistent-cache-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-worker-processes-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-logging-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-jvm-services-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-process-services-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-resources-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-messaging-6.0.1.jar 2024-05-09T20:29:13.433 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-native-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-groovy-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-base-services-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-build-option-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-cli-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-snapshots-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-files-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-kotlin-dsl-tooling-models-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-compiler-embeddable-1.3.50-patched-for-gradle-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-hashing-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-pineapple-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/gradle-docs-6.0.1.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/groovy-all-1.3-2.5.8.jar 2024-05-09T20:29:13.434 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jul-to-slf4j-1.7.28.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/log4j-over-slf4j-1.7.28.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jcl-over-slf4j-1.7.28.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/slf4j-api-1.7.28.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/guava-27.1-android.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-io-2.6.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-lang-2.6.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-commons-7.1.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-analysis-7.1.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-tree-7.1.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/asm-7.1.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-1.10.7.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk8-1.3.50.jar 2024-05-09T20:29:13.435 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/javax.inject-1.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-common-1.3.50.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-1.3.50.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-jvm-host-embeddable-1.3.50.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-embeddable-1.3.50.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-sam-with-receiver-compiler-plugin-1.3.50.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlinx-metadata-jvm-0.1.0.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/commons-compress-1.19.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/fastutil-8.3.0-min.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jansi-1.18.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/jsr305-3.0.2.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kryo-2.24.0.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-0.18.jar 2024-05-09T20:29:13.436 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/objenesis-2.6.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/trove4j-1.0.20181211.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/xml-apis-1.4.01.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-reflect-1.3.50.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-jdk7-1.3.50.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-1.3.50.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/annotations-13.0.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/ant-launcher-1.10.7.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/minlog-1.2.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-script-runtime-1.3.50.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-daemon-embeddable-1.3.50.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/failureaccess-1.0.1.jar 2024-05-09T20:29:13.437 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/kotlin-stdlib-common-1.3.50.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libcpp-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-amd64-libstdcpp-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libcpp-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-freebsd-i386-libstdcpp-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-osx-amd64-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-amd64-min-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-windows-i386-min-0.18.jar 2024-05-09T20:29:13.438 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-aarch64-ncurses5-0.18.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses5-0.18.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-amd64-ncurses6-0.18.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses5-0.18.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/native-platform-linux-i386-ncurses6-0.18.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: creating: /opt/gradle/gradle-6.0.1/lib/plugins/ 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-antlr-6.0.1.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-code-quality-6.0.1.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-signing-6.0.1.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-native-6.0.1.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-native-6.0.1.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-native-6.0.1.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-native-6.0.1.jar 2024-05-09T20:29:13.439 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-provider-plugins-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-development-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-maven-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-play-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ide-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-scala-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ear-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-play-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-javascript-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-ivy-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-jacoco-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-init-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-kotlin-dsl-tooling-builders-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugins-6.0.1.jar 2024-05-09T20:29:13.440 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-test-kit-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-groovy-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-scala-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-junit-platform-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-tooling-api-builders-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-jvm-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-java-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-language-jvm-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-jvm-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-native-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-diagnostics-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-publish-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-testing-base-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-platform-base-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-plugin-use-6.0.1.jar 2024-05-09T20:29:13.441 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-instant-execution-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-version-control-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-dependency-management-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-composite-builds-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-reporting-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-s3-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-gcs-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-cache-http-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-http-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-resources-sftp-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-build-profile-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gradle-workers-6.0.1.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/org.eclipse.jgit-5.5.0.201909110433-r.jar 2024-05-09T20:29:13.442 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-3.0.0.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-http-shared-3.0.0.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-services-storage-v1-rev136-1.25.0.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-api-client-1.25.0.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/asm-util-7.1.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ivy-2.3.0.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-compat-3.0.4.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-core-3.0.4.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-common-0.8-20100325.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/pmaven-groovy-0.8-20100325.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-file-3.0.0.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-container-default-1.7.1.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-connector-wagon-1.13.1.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpg-jdk15on-1.63.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/rhino-1.7.10.jar 2024-05-09T20:29:13.443 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/gson-2.8.5.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/simple-4.1.21.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jatl-0.2.3.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-plugin-api-3.0.4.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-oauth-client-1.25.0.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-jackson2-1.25.0.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/google-http-client-1.25.0.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/testng-6.3.1.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/snakeyaml-1.17.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/dd-plist-1.21.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-4.12.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bsh-2.0b6.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-launcher-1.3.1.jar 2024-05-09T20:29:13.444 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-s3-1.11.633.jar 2024-05-09T20:29:13.445 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-kms-1.11.633.jar 2024-05-09T20:29:13.445 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aws-java-sdk-core-1.11.633.jar 2024-05-09T20:29:13.445 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpclient-4.5.10.jar 2024-05-09T20:29:13.446 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcifs-1.3.17.jar 2024-05-09T20:29:13.446 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/nekohtml-1.9.22.jar 2024-05-09T20:29:13.451 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xercesImpl-2.12.0.jar 2024-05-09T20:29:13.453 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jsch-0.1.55.jar 2024-05-09T20:29:13.459 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jaxb-impl-2.3.2.jar 2024-05-09T20:29:13.460 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jmespath-java-1.11.633.jar 2024-05-09T20:29:13.468 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-databind-2.9.10.jar 2024-05-09T20:29:13.470 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-core-2.9.10.jar 2024-05-09T20:29:13.471 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jackson-annotations-2.9.10.jar 2024-05-09T20:29:13.475 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/joda-time-2.10.4.jar 2024-05-09T20:29:13.481 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcpkix-jdk15on-1.61.jar 2024-05-09T20:29:13.511 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/bcprov-jdk15on-1.63.jar 2024-05-09T20:29:13.513 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/commons-codec-1.13.jar 2024-05-09T20:29:13.513 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-aether-provider-3.0.4.jar 2024-05-09T20:29:13.514 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-impl-1.13.1.jar 2024-05-09T20:29:13.515 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-util-1.13.1.jar 2024-05-09T20:29:13.515 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-spi-1.13.1.jar 2024-05-09T20:29:13.516 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/aether-api-1.13.1.jar 2024-05-09T20:29:13.516 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/hamcrest-core-1.3.jar 2024-05-09T20:29:13.518 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/httpcore-4.4.12.jar 2024-05-09T20:29:13.519 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jcommander-1.72.jar 2024-05-09T20:29:13.519 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/jzlib-1.1.3.jar 2024-05-09T20:29:13.520 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-artifact-3.0.4.jar 2024-05-09T20:29:13.521 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-builder-3.0.4.jar 2024-05-09T20:29:13.522 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-model-3.0.4.jar 2024-05-09T20:29:13.522 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-repository-metadata-3.0.4.jar 2024-05-09T20:29:13.523 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-builder-3.0.4.jar 2024-05-09T20:29:13.523 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/maven-settings-3.0.4.jar 2024-05-09T20:29:13.523 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-sec-dispatcher-1.3.jar 2024-05-09T20:29:13.524 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-cipher-1.7.jar 2024-05-09T20:29:13.524 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-classworlds-2.5.1.jar 2024-05-09T20:29:13.524 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-component-annotations-1.5.5.jar 2024-05-09T20:29:13.524 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-interpolation-1.14.jar 2024-05-09T20:29:13.529 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/wagon-provider-api-3.0.0.jar 2024-05-09T20:29:13.529 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/plexus-utils-3.1.0.jar 2024-05-09T20:29:13.529 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/xbean-reflect-3.7.jar 2024-05-09T20:29:13.531 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/ion-java-1.0.2.jar 2024-05-09T20:29:13.532 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-engine-1.3.1.jar 2024-05-09T20:29:13.533 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/junit-platform-commons-1.3.1.jar 2024-05-09T20:29:13.533 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/apiguardian-api-1.0.0.jar 2024-05-09T20:29:13.533 INFO:teuthology.orchestra.run.smithi063.stdout: inflating: /opt/gradle/gradle-6.0.1/lib/plugins/opentest4j-1.1.1.jar 2024-05-09T20:29:13.568 INFO:teuthology.orchestra.run.smithi063.stdout:export PATH=/opt/gradle/gradle-6.0.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:Welcome to Gradle 6.0.1! 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout:Here are the highlights of this release: 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: - Substantial improvements in dependency management, including 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: - Publishing Gradle Module Metadata in addition to pom.xml 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: - Advanced control of transitive versions 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: - Support for optional features and dependencies 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: - Rules to tweak published metadata 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: - Support for Java 13 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: - Faster incremental Java and Groovy compilation 2024-05-09T20:29:14.011 INFO:teuthology.orchestra.run.smithi063.stdout: - New Zinc compiler for Scala 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout: - VS2019 support 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout: - Support for Gradle Enterprise plugin 3.0 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:For more details see https://docs.gradle.org/6.0.1/release-notes.html 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:------------------------------------------------------------ 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:Gradle 6.0.1 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:------------------------------------------------------------ 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:Build time: 2019-11-18 20:25:01 UTC 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout:Revision: fad121066a68c4701acd362daf4287a7c309a0f5 2024-05-09T20:29:14.012 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.013 INFO:teuthology.orchestra.run.smithi063.stdout:Kotlin: 1.3.50 2024-05-09T20:29:14.013 INFO:teuthology.orchestra.run.smithi063.stdout:Groovy: 2.5.8 2024-05-09T20:29:14.013 INFO:teuthology.orchestra.run.smithi063.stdout:Ant: Apache Ant(TM) version 1.10.7 compiled on September 1 2019 2024-05-09T20:29:14.013 INFO:teuthology.orchestra.run.smithi063.stdout:JVM: 11.0.17 (Ubuntu 11.0.17+8-post-Ubuntu-1ubuntu222.04) 2024-05-09T20:29:14.014 INFO:teuthology.orchestra.run.smithi063.stdout:OS: Linux 5.15.0-106-generic amd64 2024-05-09T20:29:14.014 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.020 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2024-05-09T20:29:14.020 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-09T20:29:14.020 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi063.front.sepia.ceph.com 2024-05-09T20:29:14.020 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-09T20:29:14.020 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi120.front.sepia.ceph.com 2024-05-09T20:29:14.020 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2024-05-09T20:29:14.203 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 'access_secret': None, 'region': 'mexico', 'endpoint': None, 'port': None, 'display_name': None, 'email': None, 'is_secure': None, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data'}} 2024-05-09T20:29:14.204 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2024-05-09T20:29:14.204 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'VHQFEIOSGWCLBSEATCCP', 'access_secret': 'wDrM2Z4NOC13lGfUHv09bVZ2egpgM+KI+l/88CfbTNYl+Sb9srVwuQ==', 'region': 'mexico', 'endpoint': 'smithi063.front.sepia.ceph.com:80', 'port': 80, 'display_name': 'Ms. tester.client.0', 'email': 'tester.client.0_test@test.test', 'is_secure': False, 'SSE': 'AES256', 's3dir': 'folder', 'dir': '../data', 'user_id': 'tester.client.0', 'host': 'smithi063.front.sepia.ceph.com'} 2024-05-09T20:29:14.204 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2024-05-09T20:29:14.204 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'VHQFEIOSGWCLBSEATCCP', '--secret', 'wDrM2Z4NOC13lGfUHv09bVZ2egpgM+KI+l/88CfbTNYl+Sb9srVwuQ==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2024-05-09T20:29:14.204 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key VHQFEIOSGWCLBSEATCCP --secret wDrM2Z4NOC13lGfUHv09bVZ2egpgM+KI+l/88CfbTNYl+Sb9srVwuQ== --email tester.client.0_test@test.test --cluster ceph 2024-05-09T20:29:14.244 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:29:14.244 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:29:14.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.291+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 realm 2024-05-09T20:29:14.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:29:14.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:29:14.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:29:14.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.295+0000 7fd08b9d2980 20 searching for the correct realm 2024-05-09T20:29:14.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.299+0000 7fd089ce7640 0 -- 172.21.15.63:0/3282447111 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fd0740ec040 msgr2=0x7fd0740d9410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:29:14.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.307+0000 7fd088ce5640 0 -- 172.21.15.63:0/3282447111 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fd07c037020 msgr2=0x7fd07c063a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:29:14.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.315+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:29:14.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 zone default found 2024-05-09T20:29:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 4 Realm: () 2024-05-09T20:29:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:29:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:29:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:29:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 zonegroup default 2024-05-09T20:29:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 started sync module instance, tier type = 2024-05-09T20:29:14.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.319+0000 7fd08b9d2980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:29:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 20 add_watcher() i=0 2024-05-09T20:29:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 20 add_watcher() i=1 2024-05-09T20:29:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 20 add_watcher() i=2 2024-05-09T20:29:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 20 add_watcher() i=3 2024-05-09T20:29:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 20 add_watcher() i=4 2024-05-09T20:29:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 20 add_watcher() i=5 2024-05-09T20:29:14.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 20 add_watcher() i=6 2024-05-09T20:29:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 20 add_watcher() i=7 2024-05-09T20:29:14.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.335+0000 7fd08b9d2980 2 all 8 watchers are set, enabling cache 2024-05-09T20:29:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fcf3f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:29:14.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:29:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:29:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 5 note: GC not initialized 2024-05-09T20:29:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fcf3e7e4640 20 reqs_thread_entry: start 2024-05-09T20:29:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 init_complete bucket index max shards: 11 2024-05-09T20:29:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 Filter name: none 2024-05-09T20:29:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fceaffff640 20 reqs_thread_entry: start 2024-05-09T20:29:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2024-05-09T20:29:14.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2024-05-09T20:29:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-05-09T20:29:14.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 cache get: name=default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP : miss 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 cache put: name=default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP info.flags=0x0 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 adding default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP to cache LRU end 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 cache get: name=default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP : hit (negative entry) 2024-05-09T20:29:14.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.339+0000 7fd08b9d2980 10 cache get: name=default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP : hit (negative entry) 2024-05-09T20:29:14.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.343+0000 7fd08b9d2980 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-05-09T20:29:14.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.343+0000 7fd08b9d2980 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-05-09T20:29:14.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.343+0000 7fd08b9d2980 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:users.uid:tester.client.0, ofs0, ns] 2024-05-09T20:29:14.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.343+0000 7fcf6cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431615 cookie 94278591556736 notifier 6944 bl.length()=670 2024-05-09T20:29:14.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.343+0000 7fcf6cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2024-05-09T20:29:14.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.343+0000 7fcf6cff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2024-05-09T20:29:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.347+0000 7fd08b9d2980 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-05-09T20:29:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.347+0000 7fd08b9d2980 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-05-09T20:29:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.347+0000 7fd08b9d2980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:tester.client.0_test@test.test, ofs0, ns] 2024-05-09T20:29:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.347+0000 7fcf6cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431588 cookie 94278591542224 notifier 6944 bl.length()=200 2024-05-09T20:29:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.347+0000 7fcf6cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2024-05-09T20:29:14.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.347+0000 7fcf6cff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2024-05-09T20:29:14.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.351+0000 7fd08b9d2980 10 cache put: name=default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP info.flags=0x7 2024-05-09T20:29:14.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.351+0000 7fd08b9d2980 10 moving default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP to cache LRU end 2024-05-09T20:29:14.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.351+0000 7fd08b9d2980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:VHQFEIOSGWCLBSEATCCP, ofs0, ns] 2024-05-09T20:29:14.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.351+0000 7fcf5bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431601 cookie 94278591545312 notifier 6944 bl.length()=189 2024-05-09T20:29:14.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.351+0000 7fcf5bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP info.flags=0x7 2024-05-09T20:29:14.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.351+0000 7fcf5bfff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+VHQFEIOSGWCLBSEATCCP to cache LRU end 2024-05-09T20:29:14.346 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:29:14.346 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "tester.client.0", 2024-05-09T20:29:14.346 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Ms. tester.client.0", 2024-05-09T20:29:14.346 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "tester.client.0_test@test.test", 2024-05-09T20:29:14.346 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:29:14.346 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "tester.client.0", 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "VHQFEIOSGWCLBSEATCCP", 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "wDrM2Z4NOC13lGfUHv09bVZ2egpgM+KI+l/88CfbTNYl+Sb9srVwuQ==", 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:29:14.345053Z" 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:29:14.347 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:29:14.348 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "rgw", 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "", 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:29:14.345033Z", 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:29:14.349 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:14.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.355+0000 7fcf3f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:29:14.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.359+0000 7fd08b9d2980 20 remove_watcher() i=0 2024-05-09T20:29:14.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.359+0000 7fd08b9d2980 2 removed watcher, disabling cache 2024-05-09T20:29:14.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.363+0000 7fd08b9d2980 20 remove_watcher() i=1 2024-05-09T20:29:14.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.363+0000 7fd08b9d2980 20 remove_watcher() i=2 2024-05-09T20:29:14.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.367+0000 7fd08b9d2980 20 remove_watcher() i=3 2024-05-09T20:29:14.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.367+0000 7fd08b9d2980 20 remove_watcher() i=4 2024-05-09T20:29:14.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.371+0000 7fd08b9d2980 20 remove_watcher() i=5 2024-05-09T20:29:14.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.371+0000 7fd08b9d2980 20 remove_watcher() i=6 2024-05-09T20:29:14.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.371+0000 7fd08b9d2980 20 remove_watcher() i=7 2024-05-09T20:29:14.380 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:29:14.380 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2024-05-09T20:29:14.432 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2024-05-09T20:29:14.432 DEBUG:teuthology.orchestra.run.smithi063:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2024-05-09T20:29:14.478 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:29:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:29:14.556 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:29:14.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.795+0000 7f9ebd328640 0 --2- 172.21.15.63:0/597661085 >> [v2:172.21.15.63:3301/0,v1:172.21.15.63:6790/0] conn(0x55e5ee23c8e0 0x55e5ee32d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-09T20:29:14.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.819+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 realm 2024-05-09T20:29:14.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:29:14.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:14.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:29:14.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.823+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:14.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.827+0000 7f9ebf013980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:29:14.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.827+0000 7f9ebf013980 20 searching for the correct realm 2024-05-09T20:29:14.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:14.827+0000 7f9ebc326640 0 -- 172.21.15.63:0/2244686188 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f9eac02b040 msgr2=0x7f9eac02c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:29:15.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.039+0000 7f9ebc326640 0 -- 172.21.15.63:0/2244686188 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f9eac01a400 msgr2=0x7f9eac01a860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:29:15.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:29:15.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:29:15.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:29:15.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:29:15.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:29:15.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:29:15.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:15.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.263+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:29:15.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:29:15.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 zone default found 2024-05-09T20:29:15.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 4 Realm: () 2024-05-09T20:29:15.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:29:15.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:29:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:29:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 zonegroup default 2024-05-09T20:29:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:15.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:15.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 started sync module instance, tier type = 2024-05-09T20:29:15.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.267+0000 7f9ebf013980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:29:15.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 20 add_watcher() i=0 2024-05-09T20:29:15.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 20 add_watcher() i=1 2024-05-09T20:29:15.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 20 add_watcher() i=2 2024-05-09T20:29:15.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 20 add_watcher() i=3 2024-05-09T20:29:15.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 20 add_watcher() i=4 2024-05-09T20:29:15.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 20 add_watcher() i=5 2024-05-09T20:29:15.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 20 add_watcher() i=6 2024-05-09T20:29:15.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 20 add_watcher() i=7 2024-05-09T20:29:15.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.295+0000 7f9ebf013980 2 all 8 watchers are set, enabling cache 2024-05-09T20:29:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.299+0000 7f9d737e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:29:15.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.303+0000 7f9ebf013980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:29:15.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.303+0000 7f9ebf013980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:29:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.391+0000 7f9d727e4640 20 reqs_thread_entry: start 2024-05-09T20:29:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.391+0000 7f9ebf013980 20 init_complete bucket index max shards: 11 2024-05-09T20:29:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.391+0000 7f9ebf013980 20 Filter name: none 2024-05-09T20:29:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.391+0000 7f9d6b7fe640 20 reqs_thread_entry: start 2024-05-09T20:29:15.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.391+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:29:15.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:29:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_SllOnU4zdLyyMQJMPprNV36hT0qkTxE', time=2024-05-09T22:24:39.859339+0000, chain.objs.size()=2 2024-05-09T20:29:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.NepXrdEsrS37EwU3YFmQ_g2KcSKdpH__0 2024-05-09T20:29:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.NepXrdEsrS37EwU3YFmQ_g2KcSKdpH__1 2024-05-09T20:29:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.2657146615741495147', time=2024-05-09T22:24:40.000239+0000, chain.objs.size()=3 2024-05-09T20:29:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-STANDARD.2~iRwd8HoES5HWxG0iadvGkVD-XMW16Wg.1 2024-05-09T20:29:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-STANDARD.2~iRwd8HoES5HWxG0iadvGkVD-XMW16Wg.2 2024-05-09T20:29:15.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.395+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-STANDARD.2~iRwd8HoES5HWxG0iadvGkVD-XMW16Wg.3 2024-05-09T20:29:15.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.403+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.411+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:29:15.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.411+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.415+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.415+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.415+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1_42e85ee79d68372dee85245f7e5cb585', time=2024-05-09T22:24:39.734445+0000, chain.objs.size()=1 2024-05-09T20:29:15.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.415+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1 2024-05-09T20:29:15.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.419+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.423+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:29:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:29:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9417961423076594371', time=2024-05-09T22:24:39.695739+0000, chain.objs.size()=1 2024-05-09T20:29:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.00Y-Tu8a-Wbc6HDK7lxrtNyFi7kzeCz_1 2024-05-09T20:29:15.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15786862865251314352', time=2024-05-09T22:24:39.869644+0000, chain.objs.size()=3 2024-05-09T20:29:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.1 2024-05-09T20:29:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2 2024-05-09T20:29:15.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.427+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.3 2024-05-09T20:29:15.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.431+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.435+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:29:15.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.439+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:29:15.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.432569573153988646', time=2024-05-09T22:24:39.529132+0000, chain.objs.size()=4 2024-05-09T20:29:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10__multipart_foo.2~ZC3Rdrn4pRIE7LzOVRrqBYqxlSz8YIS.1 2024-05-09T20:29:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10__multipart_foo.2~ZC3Rdrn4pRIE7LzOVRrqBYqxlSz8YIS.2 2024-05-09T20:29:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10__multipart_foo.2~ZC3Rdrn4pRIE7LzOVRrqBYqxlSz8YIS.3 2024-05-09T20:29:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10__multipart_foo.2~ZC3Rdrn4pRIE7LzOVRrqBYqxlSz8YIS.4 2024-05-09T20:29:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_KiAzbfH57ywPnGbhYvUHWlrJAn5nC57', time=2024-05-09T22:24:39.821335+0000, chain.objs.size()=1 2024-05-09T20:29:15.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.443+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.axjOseD9_5CgGQN_u-Nm0ZcacfKWtee_0 2024-05-09T20:29:15.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.451+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.455+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:29:15.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.459+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.459+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.463+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.463+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:29:15.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:29:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2_8494cc01ca0c51c8ecc9c910e4f7d676', time=2024-05-09T22:24:39.911024+0000, chain.objs.size()=3 2024-05-09T20:29:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.1 2024-05-09T20:29:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2 2024-05-09T20:29:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.3 2024-05-09T20:29:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17973264292651109368', time=2024-05-09T22:24:40.011144+0000, chain.objs.size()=3 2024-05-09T20:29:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-LUKEWARM.2~JEfPmSaHqZPRt0JGI2OsSBRqZsFhW-f.1 2024-05-09T20:29:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-LUKEWARM.2~JEfPmSaHqZPRt0JGI2OsSBRqZsFhW-f.2 2024-05-09T20:29:15.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.467+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-LUKEWARM.2~JEfPmSaHqZPRt0JGI2OsSBRqZsFhW-f.3 2024-05-09T20:29:15.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.475+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.479+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:29:15.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.483+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.483+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.487+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.487+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:29:15.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.491+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.491+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.495+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.495+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11107501496804072710', time=2024-05-09T22:24:39.838557+0000, chain.objs.size()=3 2024-05-09T20:29:15.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.495+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.1 2024-05-09T20:29:15.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.495+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2 2024-05-09T20:29:15.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.495+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.3 2024-05-09T20:29:15.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.495+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.503+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:29:15.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.503+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.497 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.507+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.507+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.507+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:29:15.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.511+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.515+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.515+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.515+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2_24e4605941622e1c8ec9e4771ba59838', time=2024-05-09T22:24:39.901284+0000, chain.objs.size()=3 2024-05-09T20:29:15.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.515+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.1 2024-05-09T20:29:15.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.515+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.2 2024-05-09T20:29:15.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.515+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-STANDARD.2~ZBY_M96h4H05G_2deR2DPsVM5P2Hkp3.3 2024-05-09T20:29:15.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.519+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.523+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:29:15.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.527+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.527+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.527+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:29:15.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.527+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_-7foI7HZi_DqZa1tvySSpe86XRmRuGy', time=2024-05-09T22:24:39.756102+0000, chain.objs.size()=1 2024-05-09T20:29:15.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.527+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.bxFinJddf3V7p0r-wwn2W42mScLj4jJ_0 2024-05-09T20:29:15.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.531+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8681860532371221348', time=2024-05-09T22:24:40.071209+0000, chain.objs.size()=2 2024-05-09T20:29:15.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.531+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13__shadow_.6KpARd4yTGR18HEhKqAoThCxMgGj6J9_0 2024-05-09T20:29:15.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.531+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13__shadow_.6KpARd4yTGR18HEhKqAoThCxMgGj6J9_1 2024-05-09T20:29:15.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.535+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.543+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:29:15.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_PdWtTnQ4KJNzok_4oO9uC5w0Qr0wwKY', time=2024-05-09T22:24:39.675493+0000, chain.objs.size()=1 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.OR8P3TVuvzXG1EYzlvRZDkCbXxnmiRp_0 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_X7Kom-jvycD1c3G5Bcw7md2QMH0ANoj', time=2024-05-09T22:24:39.828070+0000, chain.objs.size()=2 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.lglYukseiKAFeOr7UdB7nDb6A3SM6bj_1 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.lglYukseiKAFeOr7UdB7nDb6A3SM6bj_2 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_Ad7Xqyy-25c_rN7GA5OU8Jlo9gioJUK', time=2024-05-09T22:24:39.879834+0000, chain.objs.size()=2 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.-kcz68wsn5PSN-GbyWRTmupWoY-gbig_1 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.-kcz68wsn5PSN-GbyWRTmupWoY-gbig_2 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.70945469219796654', time=2024-05-09T22:24:39.890635+0000, chain.objs.size()=3 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.1 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.2 2024-05-09T20:29:15.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.547+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-LUKEWARM.2~JUDOV0R6c-rN-6jHar0QeYOSQdvFl4g.3 2024-05-09T20:29:15.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.559+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.563+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:29:15.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.567+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.567+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.567+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.571+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:29:15.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.571+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.575+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.575+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.575+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:29:15.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.579+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.579+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.579+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.583+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:29:15.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.587+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.587+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.587+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.587+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_fZReh8AMdjyKSLvk2pi3Tel9w06FYH5', time=2024-05-09T22:24:39.712731+0000, chain.objs.size()=1 2024-05-09T20:29:15.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.587+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Pwtl3naOzaDUz42V21L2lmu7uDx1UxX_1 2024-05-09T20:29:15.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.591+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.595+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:29:15.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.599+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.599+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.599+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.603+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:29:15.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.603+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.607+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.607+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.607+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_hMqa3qCqqdaHnFtNF6aVPi0Px9xwQL2', time=2024-05-09T22:24:39.685075+0000, chain.objs.size()=1 2024-05-09T20:29:15.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.607+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.IBiNlwP_mxP5o_7nKn647qQ0QewT1UJ_0 2024-05-09T20:29:15.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.611+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.619+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:29:15.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.623+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.623+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.623+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.623+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13676252935118820155', time=2024-05-09T22:24:40.086565+0000, chain.objs.size()=2 2024-05-09T20:29:15.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.623+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13__shadow_.XUSNQwpZdMvb2Bb90ia6m6miTR_QQ2t_0 2024-05-09T20:29:15.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.623+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13__shadow_.XUSNQwpZdMvb2Bb90ia6m6miTR_QQ2t_1 2024-05-09T20:29:15.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.631+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.639+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:29:15.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.643+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.643+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16571415963977778297', time=2024-05-09T22:24:39.422711+0000, chain.objs.size()=4 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11__multipart_foo.2~4zRieq9ilwqK3XS8IwC6qLWIeNzb5e6.1 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11__multipart_foo.2~4zRieq9ilwqK3XS8IwC6qLWIeNzb5e6.2 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11__multipart_foo.2~4zRieq9ilwqK3XS8IwC6qLWIeNzb5e6.3 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11__multipart_foo.2~4zRieq9ilwqK3XS8IwC6qLWIeNzb5e6.4 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15408003092332821369', time=2024-05-09T22:24:39.669573+0000, chain.objs.size()=1 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_0 2024-05-09T20:29:15.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_1_496317b00c9032648e6e6ec9882ef38a', time=2024-05-09T22:24:39.745927+0000, chain.objs.size()=1 2024-05-09T20:29:15.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.647+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_0 2024-05-09T20:29:15.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.651+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.655+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:29:15.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.659+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.663+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.663+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.663+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_boKDksGCANDFfLfNxv2ljhfOKWKmJjx', time=2024-05-09T22:24:39.848879+0000, chain.objs.size()=2 2024-05-09T20:29:15.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.663+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.tUqJDp0FiphJZ62UGdNG8m_37Jgmi2R_0 2024-05-09T20:29:15.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.663+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.tUqJDp0FiphJZ62UGdNG8m_37Jgmi2R_1 2024-05-09T20:29:15.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.671+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.675+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:29:15.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.679+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.683+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.683+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.687+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:29:15.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.687+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.691+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.691+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.691+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10979833608485621736', time=2024-05-09T22:24:39.718526+0000, chain.objs.size()=1 2024-05-09T20:29:15.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.691+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.uZQuqQeEpXW3Rz5y3gnRxEomIRWwTEp_0 2024-05-09T20:29:15.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.695+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.699+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:29:15.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.703+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.703+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.707+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.707+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_-nC3P-id3d4pA4w6x3ZNYqrqc3pzYfu', time=2024-05-09T22:24:39.645573+0000, chain.objs.size()=1 2024-05-09T20:29:15.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.707+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.-BJM1WK2CeZLTtbkjYXSk7T1TRlWlRp_0 2024-05-09T20:29:15.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.711+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.715+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:29:15.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.719+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8842038008404339350', time=2024-05-09T22:24:39.591598+0000, chain.objs.size()=4 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9__multipart_foo.2~pmYrgdQUmJNVSxioqlbeMyu3_FtSBdd.1 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9__multipart_foo.2~pmYrgdQUmJNVSxioqlbeMyu3_FtSBdd.2 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9__multipart_foo.2~pmYrgdQUmJNVSxioqlbeMyu3_FtSBdd.3 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9__multipart_foo.2~pmYrgdQUmJNVSxioqlbeMyu3_FtSBdd.4 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2_6c4622c91d7effb4b3496e9f7f872839', time=2024-05-09T22:24:39.936442+0000, chain.objs.size()=3 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.1 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.2 2024-05-09T20:29:15.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.723+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__multipart_foo-FROZEN.2~gh5BwElhC5Yp7VYPD6FduEjMqhzyFyZ.3 2024-05-09T20:29:15.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.731+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.735+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:29:15.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.739+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.739+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.743+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.743+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:29:15.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.747+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.747+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.747+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.751+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:29:15.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.755+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.755+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.759+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:29:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.759+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_fEyVgs89_z0CBqYk48IpiWhPAvqvtQY', time=2024-05-09T22:24:39.926306+0000, chain.objs.size()=1 2024-05-09T20:29:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.759+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16__shadow_.nxA66Srw8M-PCSXd8BURxC_LpT7MZKM_0 2024-05-09T20:29:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.759+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.60331768207756217', time=2024-05-09T22:24:39.993519+0000, chain.objs.size()=3 2024-05-09T20:29:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.759+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-FROZEN.2~x0I2e2ukALgDWftEc_XFCDuQBc_Wl3R.1 2024-05-09T20:29:15.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.759+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-FROZEN.2~x0I2e2ukALgDWftEc_XFCDuQBc_Wl3R.2 2024-05-09T20:29:15.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.759+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14__multipart_foo-FROZEN.2~x0I2e2ukALgDWftEc_XFCDuQBc_Wl3R.3 2024-05-09T20:29:15.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.763+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.771+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:29:15.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.771+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.775+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.775+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.775+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9088638616493081612', time=2024-05-09T22:24:39.467534+0000, chain.objs.size()=4 2024-05-09T20:29:15.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.775+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12__multipart_foo.2~Jf0RyadJWJv1w8WUBA6Yl-L1arrwOTM.1 2024-05-09T20:29:15.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.775+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12__multipart_foo.2~Jf0RyadJWJv1w8WUBA6Yl-L1arrwOTM.2 2024-05-09T20:29:15.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.775+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12__multipart_foo.2~Jf0RyadJWJv1w8WUBA6Yl-L1arrwOTM.3 2024-05-09T20:29:15.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.775+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12__multipart_foo.2~Jf0RyadJWJv1w8WUBA6Yl-L1arrwOTM.4 2024-05-09T20:29:15.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.779+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.787+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:29:15.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.791+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.791+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.791+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.791+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_1_602aad55e8bcbd9d8ec08ab063bca4ee', time=2024-05-09T22:24:39.761256+0000, chain.objs.size()=1 2024-05-09T20:29:15.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.791+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Xu3pHJw9pL5Krhtlq4NeuxUgJ9peYWS_0 2024-05-09T20:29:15.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.795+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.799+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:29:15.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.803+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.807+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.807+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.811+0000 7f9ebf013980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:29:15.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.815+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:15.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.819+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:15.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.819+0000 7f9ebf013980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:29:15.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.819+0000 7f9ebf013980 20 garbage collection: RGWGC::process iterating over entry tag='_IMs_70PI-3ORMXRRqmA_ArTEujnJP5c', time=2024-05-09T22:24:39.657470+0000, chain.objs.size()=1 2024-05-09T20:29:15.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.819+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15__shadow_.Wig8qH7dQ6JdvzDwjNl0SqCLpqWsph9_1 2024-05-09T20:29:15.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.823+0000 7f9ebf013980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:29:15.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.831+0000 7f9d737e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:29:15.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.839+0000 7f9ebf013980 20 remove_watcher() i=0 2024-05-09T20:29:15.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.839+0000 7f9ebf013980 2 removed watcher, disabling cache 2024-05-09T20:29:15.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.839+0000 7f9ebf013980 20 remove_watcher() i=1 2024-05-09T20:29:15.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.843+0000 7f9ebf013980 20 remove_watcher() i=2 2024-05-09T20:29:15.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.843+0000 7f9ebf013980 20 remove_watcher() i=3 2024-05-09T20:29:15.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.847+0000 7f9ebf013980 20 remove_watcher() i=4 2024-05-09T20:29:15.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.851+0000 7f9ebf013980 20 remove_watcher() i=5 2024-05-09T20:29:15.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.851+0000 7f9ebf013980 20 remove_watcher() i=6 2024-05-09T20:29:15.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.855+0000 7f9ebf013980 20 remove_watcher() i=7 2024-05-09T20:29:15.866 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:29:15.940 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:29:15.941 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:29:15.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.995+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:15.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 realm 2024-05-09T20:29:15.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:15.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:29:15.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:15.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:29:15.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:15.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:29:15.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:15.999+0000 7f4700a4c980 20 searching for the correct realm 2024-05-09T20:29:15.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.003+0000 7f46fed61640 0 -- 172.21.15.63:0/295234126 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f46f402b040 msgr2=0x7f46f40602a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:29:16.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.011+0000 7f46fdd5f640 0 -- 172.21.15.63:0/295234126 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f46f0027040 msgr2=0x7f46f0027420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:29:16.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:29:16.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:29:16.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:29:16.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:29:16.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:29:16.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:29:16.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:16.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:16.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:16.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:29:16.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.019+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 zone default found 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 4 Realm: () 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 zonegroup default 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:16.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:16.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:16.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 rados->read ofs=0 len=0 2024-05-09T20:29:16.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:29:16.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 started sync module instance, tier type = 2024-05-09T20:29:16.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.023+0000 7f4700a4c980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:29:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 20 add_watcher() i=0 2024-05-09T20:29:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 20 add_watcher() i=1 2024-05-09T20:29:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 20 add_watcher() i=2 2024-05-09T20:29:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 20 add_watcher() i=3 2024-05-09T20:29:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 20 add_watcher() i=4 2024-05-09T20:29:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 20 add_watcher() i=5 2024-05-09T20:29:16.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 20 add_watcher() i=6 2024-05-09T20:29:16.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 20 add_watcher() i=7 2024-05-09T20:29:16.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.039+0000 7f4700a4c980 2 all 8 watchers are set, enabling cache 2024-05-09T20:29:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.043+0000 7f45b4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:29:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.043+0000 7f4700a4c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:29:16.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.043+0000 7f4700a4c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:29:16.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.115+0000 7f452f7fe640 20 reqs_thread_entry: start 2024-05-09T20:29:16.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.115+0000 7f4700a4c980 20 init_complete bucket index max shards: 11 2024-05-09T20:29:16.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.115+0000 7f4700a4c980 20 Filter name: none 2024-05-09T20:29:16.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.115+0000 7f452d7fa640 20 reqs_thread_entry: start 2024-05-09T20:29:16.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.115+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:29:16.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.119+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.119+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.123+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.123+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:29:16.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.127+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.131+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.131+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.135+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:29:16.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.135+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.139+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.139+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.143+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:29:16.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.143+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.147+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.147+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.147+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:29:16.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.151+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.151+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.155+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.159+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:29:16.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.159+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.159+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.163+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.163+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:29:16.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.167+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.167+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.167+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.171+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:29:16.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.175+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.175+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.179+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.179+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:29:16.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.183+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.187+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.187+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.191+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:29:16.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.195+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.195+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.195+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.199+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:29:16.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.203+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.203+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.207+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.207+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:29:16.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.211+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.215+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.215+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.215+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:29:16.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.219+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.223+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.223+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.223+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:29:16.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.227+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.231+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.223 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.231+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.235+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:29:16.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.239+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.239+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.239+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.243+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:29:16.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.247+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.247+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.247+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.251+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:29:16.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.255+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.255+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.255+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.259+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:29:16.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.259+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.263+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.263+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.267+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:29:16.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.267+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.271+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.271+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.275+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:29:16.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.279+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.279+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.279+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.283+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:29:16.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.287+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.291+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.291+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.295+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:29:16.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.299+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.299+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.303+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.303+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:29:16.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.307+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.311+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.311+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.315+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:29:16.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.319+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.319+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.319+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.323+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:29:16.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.327+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.327+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.331+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.331+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:29:16.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.335+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.339+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.339+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.343+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:29:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.343+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.347+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.347+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.351+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:29:16.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.355+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.355+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.355+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.359+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:29:16.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.363+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.363+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.367+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.367+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:29:16.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.371+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.375+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.375+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.379+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:29:16.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.383+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.383+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.383+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.387+0000 7f4700a4c980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:29:16.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.391+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.395+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:29:16.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.395+0000 7f4700a4c980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:29:16.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.399+0000 7f45b4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:29:16.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.403+0000 7f4700a4c980 20 remove_watcher() i=0 2024-05-09T20:29:16.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.403+0000 7f4700a4c980 2 removed watcher, disabling cache 2024-05-09T20:29:16.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.407+0000 7f4700a4c980 20 remove_watcher() i=1 2024-05-09T20:29:16.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.411+0000 7f4700a4c980 20 remove_watcher() i=2 2024-05-09T20:29:16.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.411+0000 7f4700a4c980 20 remove_watcher() i=3 2024-05-09T20:29:16.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.415+0000 7f4700a4c980 20 remove_watcher() i=4 2024-05-09T20:29:16.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.419+0000 7f4700a4c980 20 remove_watcher() i=5 2024-05-09T20:29:16.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.419+0000 7f4700a4c980 20 remove_watcher() i=6 2024-05-09T20:29:16.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:29:16.423+0000 7f4700a4c980 20 remove_watcher() i=7 2024-05-09T20:29:16.434 DEBUG:teuthology.orchestra.run.smithi063:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2024-05-09T20:29:17.060 INFO:teuthology.orchestra.run.smithi063.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2024-05-09T20:29:21.661 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :clean UP-TO-DATE 2024-05-09T20:29:42.360 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :compileJava 2024-05-09T20:29:42.360 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :processResources 2024-05-09T20:29:42.360 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :classes 2024-05-09T20:29:43.159 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:43.159 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :compileTestJava 2024-05-09T20:29:43.160 INFO:teuthology.orchestra.run.smithi063.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-05-09T20:29:43.160 INFO:teuthology.orchestra.run.smithi063.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-05-09T20:29:43.161 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:43.161 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :processTestResources NO-SOURCE 2024-05-09T20:29:43.161 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :testClasses 2024-05-09T20:29:49.261 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:49.262 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :test 2024-05-09T20:29:49.262 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:49.359 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2024-05-09T20:29:57.459 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:57.459 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2024-05-09T20:29:58.259 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:58.259 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2024-05-09T20:29:58.559 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:58.559 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2024-05-09T20:29:59.359 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:29:59.359 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2024-05-09T20:30:09.659 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:09.659 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2024-05-09T20:30:10.559 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:10.561 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2024-05-09T20:30:10.659 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:10.659 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2024-05-09T20:30:10.859 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:10.859 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2024-05-09T20:30:11.059 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:11.059 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2024-05-09T20:30:11.859 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:11.859 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2024-05-09T20:30:21.259 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:21.259 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2024-05-09T20:30:22.859 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:22.859 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2024-05-09T20:30:23.359 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:30:23.359 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STANDARD_OUT 2024-05-09T20:30:23.359 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09 20:30:23,354 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-05-09T20:31:27.260 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:27.260 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2024-05-09T20:31:28.859 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:28.859 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2024-05-09T20:31:29.059 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:29.059 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2024-05-09T20:31:29.859 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:29.859 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2024-05-09T20:31:30.159 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:30.159 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2024-05-09T20:31:31.659 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:31.659 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2024-05-09T20:31:40.259 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:40.259 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2024-05-09T20:31:41.159 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:41.159 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2024-05-09T20:31:41.259 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:41.259 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2024-05-09T20:31:41.459 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:41.459 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2024-05-09T20:31:53.259 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:53.260 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2024-05-09T20:31:54.759 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:54.759 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests STANDARD_OUT 2024-05-09T20:31:54.760 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09 20:31:54,758 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-05-09T20:31:55.559 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:31:55.559 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2024-05-09T20:32:12.359 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:32:12.360 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2024-05-09T20:32:13.559 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:32:13.559 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2024-05-09T20:32:18.259 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:32:18.259 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2024-05-09T20:32:19.059 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:32:19.059 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2024-05-09T20:34:29.859 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:34:29.860 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2024-05-09T20:34:30.959 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:34:30.959 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2024-05-09T20:34:41.559 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:34:41.559 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2024-05-09T20:34:42.359 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:34:42.359 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2024-05-09T20:34:50.759 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:34:50.759 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2024-05-09T20:34:51.559 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:34:51.559 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2024-05-09T20:34:58.059 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:34:58.059 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2024-05-09T20:35:17.860 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:17.860 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2024-05-09T20:35:18.459 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:18.460 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2024-05-09T20:35:20.660 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:20.660 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2024-05-09T20:35:20.860 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:20.860 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2024-05-09T20:35:21.760 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:21.760 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2024-05-09T20:35:22.060 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:22.060 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2024-05-09T20:35:22.960 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:22.960 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2024-05-09T20:35:23.260 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:23.260 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2024-05-09T20:35:24.160 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:24.160 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2024-05-09T20:35:24.360 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:24.360 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2024-05-09T20:35:25.360 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:25.360 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2024-05-09T20:35:25.560 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:25.560 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2024-05-09T20:35:26.460 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:26.460 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2024-05-09T20:35:26.660 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:26.660 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2024-05-09T20:35:27.561 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:27.561 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2024-05-09T20:35:47.960 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:47.961 INFO:teuthology.orchestra.run.smithi063.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2024-05-09T20:35:48.860 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:48.860 INFO:teuthology.orchestra.run.smithi063.stdout:---------------------------------------------------------------------- 2024-05-09T20:35:48.860 INFO:teuthology.orchestra.run.smithi063.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2024-05-09T20:35:48.861 INFO:teuthology.orchestra.run.smithi063.stdout:---------------------------------------------------------------------- 2024-05-09T20:35:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:48.887 INFO:teuthology.orchestra.run.smithi063.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-05-09T20:35:48.887 INFO:teuthology.orchestra.run.smithi063.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-05-09T20:35:48.887 INFO:teuthology.orchestra.run.smithi063.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-05-09T20:35:48.887 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:48.887 INFO:teuthology.orchestra.run.smithi063.stdout:BUILD SUCCESSFUL in 6m 32s 2024-05-09T20:35:48.887 INFO:teuthology.orchestra.run.smithi063.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2024-05-09T20:35:49.214 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:35:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:35:49.247 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:35:49.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 20 realm 2024-05-09T20:35:49.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:35:49.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.299+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.303+0000 7f0e95298980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:35:49.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.303+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.303+0000 7f0e95298980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:35:49.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.303+0000 7f0e95298980 20 searching for the correct realm 2024-05-09T20:35:49.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.307+0000 7f0e935ad640 0 -- 172.21.15.63:0/2878279799 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f0e880bf040 msgr2=0x7f0e8812ea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:35:49.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.315+0000 7f0e925ab640 0 -- 172.21.15.63:0/2878279799 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f0e84018020 msgr2=0x7f0e840550b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:35:49.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:35:49.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:35:49.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:35:49.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:35:49.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:35:49.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:35:49.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:35:49.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:35:49.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 zone default found 2024-05-09T20:35:49.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 4 Realm: () 2024-05-09T20:35:49.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:35:49.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:35:49.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:35:49.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 zonegroup default 2024-05-09T20:35:49.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.323+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.327+0000 7f0e95298980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.327+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.327+0000 7f0e95298980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.327+0000 7f0e95298980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.327+0000 7f0e95298980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.327+0000 7f0e95298980 20 started sync module instance, tier type = 2024-05-09T20:35:49.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.327+0000 7f0e95298980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:35:49.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 20 add_watcher() i=0 2024-05-09T20:35:49.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 20 add_watcher() i=1 2024-05-09T20:35:49.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 20 add_watcher() i=2 2024-05-09T20:35:49.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 20 add_watcher() i=3 2024-05-09T20:35:49.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 20 add_watcher() i=4 2024-05-09T20:35:49.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 20 add_watcher() i=5 2024-05-09T20:35:49.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 20 add_watcher() i=6 2024-05-09T20:35:49.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 20 add_watcher() i=7 2024-05-09T20:35:49.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.343+0000 7f0e95298980 2 all 8 watchers are set, enabling cache 2024-05-09T20:35:49.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.347+0000 7f0d497e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:35:49.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.347+0000 7f0e95298980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:35:49.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.347+0000 7f0e95298980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:35:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.423+0000 7f0cc3fff640 20 reqs_thread_entry: start 2024-05-09T20:35:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.423+0000 7f0e95298980 20 init_complete bucket index max shards: 11 2024-05-09T20:35:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.423+0000 7f0e95298980 20 Filter name: none 2024-05-09T20:35:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.423+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:35:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.423+0000 7f0cc1ffb640 20 reqs_thread_entry: start 2024-05-09T20:35:49.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.423+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.427+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.427+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.427+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1453890038604354071', time=2024-05-09T22:30:09.930708+0000, chain.objs.size()=5 2024-05-09T20:35:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.427+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.31__multipart_key1.2~cx4bof8hKr5FXvUAC5rrEu4nteeSexq.1 2024-05-09T20:35:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.427+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.31__multipart_key1.2~cx4bof8hKr5FXvUAC5rrEu4nteeSexq.2 2024-05-09T20:35:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.427+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.31__multipart_key1.2~cx4bof8hKr5FXvUAC5rrEu4nteeSexq.3 2024-05-09T20:35:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.427+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.31__multipart_key1.2~cx4bof8hKr5FXvUAC5rrEu4nteeSexq.4 2024-05-09T20:35:49.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.427+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.31__multipart_key1.2~cx4bof8hKr5FXvUAC5rrEu4nteeSexq.5 2024-05-09T20:35:49.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.431+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.435+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:35:49.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.2064719593514429530', time=2024-05-09T22:31:40.480360+0000, chain.objs.size()=5 2024-05-09T20:35:49.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.40__multipart_key1.2~R94qt1tfwXW8sDN-qd4WBfjCq871py4.1 2024-05-09T20:35:49.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.40__multipart_key1.2~R94qt1tfwXW8sDN-qd4WBfjCq871py4.2 2024-05-09T20:35:49.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.40__multipart_key1.2~R94qt1tfwXW8sDN-qd4WBfjCq871py4.3 2024-05-09T20:35:49.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.40__multipart_key1.2~R94qt1tfwXW8sDN-qd4WBfjCq871py4.4 2024-05-09T20:35:49.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.439+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.40__multipart_key1.2~R94qt1tfwXW8sDN-qd4WBfjCq871py4.5 2024-05-09T20:35:49.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.451+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.455+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:35:49.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.455+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.459+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.459+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.463+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:35:49.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.2755681240180209154', time=2024-05-09T22:33:14.181505+0000, chain.objs.size()=11 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.1 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.2 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.3 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.4 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.5 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.6 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.7 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.8 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.9 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.10 2024-05-09T20:35:49.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.467+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~kJTnhKEGWP5q5WUK76WDLGaZkediPFI.11 2024-05-09T20:35:49.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.479+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.491+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:35:49.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.491+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.491+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.485 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.491+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.495+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:35:49.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.15879753437589102645', time=2024-05-09T22:31:28.214265+0000, chain.objs.size()=5 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~zMszFXDW35u6ONqMU9mv_kgcSJRUjMS.1 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__shadow_key1.2~zMszFXDW35u6ONqMU9mv_kgcSJRUjMS.1_1 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~zMszFXDW35u6ONqMU9mv_kgcSJRUjMS.2 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__shadow_key1.2~zMszFXDW35u6ONqMU9mv_kgcSJRUjMS.2_1 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~zMszFXDW35u6ONqMU9mv_kgcSJRUjMS.3 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13501933139949106254', time=2024-05-09T22:34:41.738961+0000, chain.objs.size()=5 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.45__multipart_key1.2~eFoT--6zxms7jJYHxvEF7foybLOutaG.1 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.45__multipart_key1.2~eFoT--6zxms7jJYHxvEF7foybLOutaG.2 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.45__multipart_key1.2~eFoT--6zxms7jJYHxvEF7foybLOutaG.3 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.45__multipart_key1.2~eFoT--6zxms7jJYHxvEF7foybLOutaG.4 2024-05-09T20:35:49.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.499+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.45__multipart_key1.2~eFoT--6zxms7jJYHxvEF7foybLOutaG.5 2024-05-09T20:35:49.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.511+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.519+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:35:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14461425660741040421', time=2024-05-09T22:32:12.776474+0000, chain.objs.size()=5 2024-05-09T20:35:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file2.mpg.2~CJLFLPwepcO_rk5P3l9jKjWcLT1IQoF.1 2024-05-09T20:35:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file2.mpg.2~CJLFLPwepcO_rk5P3l9jKjWcLT1IQoF.2 2024-05-09T20:35:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file2.mpg.2~CJLFLPwepcO_rk5P3l9jKjWcLT1IQoF.3 2024-05-09T20:35:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file2.mpg.2~CJLFLPwepcO_rk5P3l9jKjWcLT1IQoF.4 2024-05-09T20:35:49.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.523+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file2.mpg.2~CJLFLPwepcO_rk5P3l9jKjWcLT1IQoF.5 2024-05-09T20:35:49.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.527+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.535+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:35:49.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.535+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.535+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='2~VkU3mogIzjuIDtN-hpotcxgUoQMRO3T', time=2024-05-09T22:29:57.350436+0000, chain.objs.size()=5 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.29__multipart_key1.2~VkU3mogIzjuIDtN-hpotcxgUoQMRO3T.1 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.29__multipart_key1.2~VkU3mogIzjuIDtN-hpotcxgUoQMRO3T.2 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.29__multipart_key1.2~VkU3mogIzjuIDtN-hpotcxgUoQMRO3T.3 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.29__multipart_key1.2~VkU3mogIzjuIDtN-hpotcxgUoQMRO3T.4 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.29__multipart_key1.2~VkU3mogIzjuIDtN-hpotcxgUoQMRO3T.5 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9071463698149112344', time=2024-05-09T22:31:03.561396+0000, chain.objs.size()=5 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~1s0jo87hdt34JeAgKhqVR5cO-PogpIG.1 2024-05-09T20:35:49.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~1s0jo87hdt34JeAgKhqVR5cO-PogpIG.2 2024-05-09T20:35:49.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~1s0jo87hdt34JeAgKhqVR5cO-PogpIG.3 2024-05-09T20:35:49.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~1s0jo87hdt34JeAgKhqVR5cO-PogpIG.4 2024-05-09T20:35:49.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~1s0jo87hdt34JeAgKhqVR5cO-PogpIG.5 2024-05-09T20:35:49.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10769795310164157973', time=2024-05-09T22:31:21.248372+0000, chain.objs.size()=5 2024-05-09T20:35:49.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~Qer7S9lPA9g3dYrGbJJzkK-metICN0g.1 2024-05-09T20:35:49.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.539+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__shadow_key1.2~Qer7S9lPA9g3dYrGbJJzkK-metICN0g.1_1 2024-05-09T20:35:49.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.543+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~Qer7S9lPA9g3dYrGbJJzkK-metICN0g.2 2024-05-09T20:35:49.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.543+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__shadow_key1.2~Qer7S9lPA9g3dYrGbJJzkK-metICN0g.2_1 2024-05-09T20:35:49.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.547+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~Qer7S9lPA9g3dYrGbJJzkK-metICN0g.3 2024-05-09T20:35:49.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.551+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.559+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:35:49.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.559+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.559+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.4547486350646057625', time=2024-05-09T22:34:29.756452+0000, chain.objs.size()=11 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.1 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.1_1 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.2 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.2_1 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.3 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.3_1 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.4 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.4_1 2024-05-09T20:35:49.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.5 2024-05-09T20:35:49.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.5_1 2024-05-09T20:35:49.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.563+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~1Weml9WCcsey8fKvG6jCD1K-hltoDDt.6 2024-05-09T20:35:49.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.571+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.575+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:35:49.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.579+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.579+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.579+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.579+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:35:49.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.583+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.583+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.583+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.587+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:35:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.587+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.587+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.591+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.591+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:35:49.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-09T20:35:49.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12484834664684608631', time=2024-05-09T22:31:10.321197+0000, chain.objs.size()=5 2024-05-09T20:35:49.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~z6u5BVZ_zCVjIN68kaGthpNjzzzCRo_.1 2024-05-09T20:35:49.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~z6u5BVZ_zCVjIN68kaGthpNjzzzCRo_.2 2024-05-09T20:35:49.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~z6u5BVZ_zCVjIN68kaGthpNjzzzCRo_.3 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~z6u5BVZ_zCVjIN68kaGthpNjzzzCRo_.4 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~z6u5BVZ_zCVjIN68kaGthpNjzzzCRo_.5 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16235573090466033900', time=2024-05-09T22:31:27.487165+0000, chain.objs.size()=4 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.35__shadow_.wXCdib3pk-k_WxYPomqhXllKJrTd9w5_1 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.35__shadow_.wXCdib3pk-k_WxYPomqhXllKJrTd9w5_2 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.35__shadow_.wXCdib3pk-k_WxYPomqhXllKJrTd9w5_3 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.35__shadow_.wXCdib3pk-k_WxYPomqhXllKJrTd9w5_4 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16475743077167895620', time=2024-05-09T22:35:48.207539+0000, chain.objs.size()=11 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.1 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.2 2024-05-09T20:35:49.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.595+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.3 2024-05-09T20:35:49.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.599+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.4 2024-05-09T20:35:49.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.599+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.5 2024-05-09T20:35:49.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.603+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.6 2024-05-09T20:35:49.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.603+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.7 2024-05-09T20:35:49.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.603+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.8 2024-05-09T20:35:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.603+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.9 2024-05-09T20:35:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.603+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.10 2024-05-09T20:35:49.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.603+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.55__multipart_key1.2~n1o5-frf92N6JWyT3h7L0JtMlWcnu73.11 2024-05-09T20:35:49.604 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.611+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.615+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:35:49.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16906284157901353719', time=2024-05-09T22:34:50.968186+0000, chain.objs.size()=5 2024-05-09T20:35:49.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.46__multipart_key1.2~hs_TnWPg8Dsyiq8PLFelwcvVOrN0nMQ.1 2024-05-09T20:35:49.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.46__multipart_key1.2~hs_TnWPg8Dsyiq8PLFelwcvVOrN0nMQ.2 2024-05-09T20:35:49.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.46__multipart_key1.2~hs_TnWPg8Dsyiq8PLFelwcvVOrN0nMQ.3 2024-05-09T20:35:49.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.46__multipart_key1.2~hs_TnWPg8Dsyiq8PLFelwcvVOrN0nMQ.4 2024-05-09T20:35:49.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.619+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.46__multipart_key1.2~hs_TnWPg8Dsyiq8PLFelwcvVOrN0nMQ.5 2024-05-09T20:35:49.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.627+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.627+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:35:49.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.631+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.631+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-09T20:35:49.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8932921217578570969', time=2024-05-09T22:30:21.511651+0000, chain.objs.size()=5 2024-05-09T20:35:49.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.1 2024-05-09T20:35:49.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.2 2024-05-09T20:35:49.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.3 2024-05-09T20:35:49.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.4 2024-05-09T20:35:49.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.5 2024-05-09T20:35:49.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6784945069474169118', time=2024-05-09T22:31:54.785242+0000, chain.objs.size()=5 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.41__multipart_key1.2~dJF90BHBvSIOERuumKyRLogG-vKhtHi.1 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.41__multipart_key1.2~dJF90BHBvSIOERuumKyRLogG-vKhtHi.2 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.41__multipart_key1.2~dJF90BHBvSIOERuumKyRLogG-vKhtHi.3 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.41__multipart_key1.2~dJF90BHBvSIOERuumKyRLogG-vKhtHi.4 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.41__multipart_key1.2~dJF90BHBvSIOERuumKyRLogG-vKhtHi.5 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp', time=2024-05-09T22:35:14.150874+0000, chain.objs.size()=1 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.20 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp', time=2024-05-09T22:35:14.442711+0000, chain.objs.size()=59 2024-05-09T20:35:49.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.635+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.1 2024-05-09T20:35:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.639+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.2 2024-05-09T20:35:49.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.639+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.3 2024-05-09T20:35:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.4 2024-05-09T20:35:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.5 2024-05-09T20:35:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.6 2024-05-09T20:35:49.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.7 2024-05-09T20:35:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.8 2024-05-09T20:35:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.9 2024-05-09T20:35:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.10 2024-05-09T20:35:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.11 2024-05-09T20:35:49.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.12 2024-05-09T20:35:49.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.643+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.13 2024-05-09T20:35:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.647+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.14 2024-05-09T20:35:49.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.647+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.15 2024-05-09T20:35:49.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.647+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.16 2024-05-09T20:35:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.647+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.17 2024-05-09T20:35:49.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.647+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.18 2024-05-09T20:35:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.651+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.19 2024-05-09T20:35:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.651+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.cITMb3xZdTOozDfeacHNi-e2skYIZPhR.20 2024-05-09T20:35:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.651+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.21 2024-05-09T20:35:49.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.651+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.22 2024-05-09T20:35:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.651+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.23 2024-05-09T20:35:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.651+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.24 2024-05-09T20:35:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.651+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.25 2024-05-09T20:35:49.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.651+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.26 2024-05-09T20:35:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.655+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.27 2024-05-09T20:35:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.655+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.28 2024-05-09T20:35:49.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.655+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.29 2024-05-09T20:35:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.655+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.30 2024-05-09T20:35:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.655+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.31 2024-05-09T20:35:49.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.655+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.32 2024-05-09T20:35:49.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.655+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.33 2024-05-09T20:35:49.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.655+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.34 2024-05-09T20:35:49.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.659+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.35 2024-05-09T20:35:49.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.659+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.36 2024-05-09T20:35:49.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.659+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.37 2024-05-09T20:35:49.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.659+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.38 2024-05-09T20:35:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.659+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.39 2024-05-09T20:35:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.659+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.40 2024-05-09T20:35:49.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.659+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.41 2024-05-09T20:35:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.42 2024-05-09T20:35:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.43 2024-05-09T20:35:49.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.44 2024-05-09T20:35:49.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.45 2024-05-09T20:35:49.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.46 2024-05-09T20:35:49.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.47 2024-05-09T20:35:49.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.48 2024-05-09T20:35:49.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.49 2024-05-09T20:35:49.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.663+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.50 2024-05-09T20:35:49.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.667+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.51 2024-05-09T20:35:49.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.667+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.52 2024-05-09T20:35:49.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.667+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.53 2024-05-09T20:35:49.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.667+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.54 2024-05-09T20:35:49.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.671+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.55 2024-05-09T20:35:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.671+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.56 2024-05-09T20:35:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.671+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.57 2024-05-09T20:35:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.671+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.58 2024-05-09T20:35:49.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.671+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.47__multipart_key1.2~-7D3tuNgo1vWWec4ylmqYKcKt9ckyJp.59 2024-05-09T20:35:49.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.675+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.679+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:35:49.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.679+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.679+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.679+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:35:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.272360155199132722', time=2024-05-09T22:33:53.897260+0000, chain.objs.size()=10 2024-05-09T20:35:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.1 2024-05-09T20:35:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.2 2024-05-09T20:35:49.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.3 2024-05-09T20:35:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.4 2024-05-09T20:35:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.5 2024-05-09T20:35:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.6 2024-05-09T20:35:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.7 2024-05-09T20:35:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.8 2024-05-09T20:35:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.9 2024-05-09T20:35:49.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.683+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~3WGL_-5zLfrX2M-KLjIktqo3PPQw0OU.10 2024-05-09T20:35:49.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.691+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.691+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:35:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.695+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.695+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.695+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.695+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:35:49.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11822474938746449290', time=2024-05-09T22:34:30.124816+0000, chain.objs.size()=11 2024-05-09T20:35:49.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.1 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.1_1 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.2 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.2_1 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.3 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.3_1 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.4 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.4_1 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.5 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.5_1 2024-05-09T20:35:49.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.699+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~ZiUa07OmrHtp8HCXbgtjDPmGgqUDPls.6 2024-05-09T20:35:49.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.707+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.707+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:35:49.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.711+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.711+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.711+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.711+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:35:49.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.3437299219749453830', time=2024-05-09T22:31:27.141137+0000, chain.objs.size()=5 2024-05-09T20:35:49.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~6FnngiBkX2OUwft2FKa_zrsuhcw9VcK.1 2024-05-09T20:35:49.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__shadow_key1.2~6FnngiBkX2OUwft2FKa_zrsuhcw9VcK.1_1 2024-05-09T20:35:49.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~6FnngiBkX2OUwft2FKa_zrsuhcw9VcK.2 2024-05-09T20:35:49.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__shadow_key1.2~6FnngiBkX2OUwft2FKa_zrsuhcw9VcK.2_1 2024-05-09T20:35:49.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.715+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~6FnngiBkX2OUwft2FKa_zrsuhcw9VcK.3 2024-05-09T20:35:49.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.719+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.719+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:35:49.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.723+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.723+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.723+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.723+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:35:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.723+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.727+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.727+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.727+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.5622595962178672964', time=2024-05-09T22:31:15.795502+0000, chain.objs.size()=5 2024-05-09T20:35:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.727+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~2z-jGL1FX195o3nGqHCQb1YSnGa813Z.1 2024-05-09T20:35:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.727+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~2z-jGL1FX195o3nGqHCQb1YSnGa813Z.2 2024-05-09T20:35:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.727+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~2z-jGL1FX195o3nGqHCQb1YSnGa813Z.3 2024-05-09T20:35:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.727+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~2z-jGL1FX195o3nGqHCQb1YSnGa813Z.4 2024-05-09T20:35:49.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.727+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.36__multipart_key1.2~2z-jGL1FX195o3nGqHCQb1YSnGa813Z.5 2024-05-09T20:35:49.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.731+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.731+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:35:49.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.735+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.735+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.735+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.735+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:35:49.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.739+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.739+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.739+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.739+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:35:49.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.739+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.2_d941c69e3a46e3720069463475f0b0c6', time=2024-05-09T22:30:22.253755+0000, chain.objs.size()=5 2024-05-09T20:35:49.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.1 2024-05-09T20:35:49.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.2 2024-05-09T20:35:49.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.3 2024-05-09T20:35:49.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.4 2024-05-09T20:35:49.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.33__multipart_key1.2~XcbvgHstAQdcTvw1_bUTOxozIKKHxpP.5 2024-05-09T20:35:49.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.743+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:35:49.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.8874292161677590681', time=2024-05-09T22:32:12.587650+0000, chain.objs.size()=5 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file.mpg.2~ETL7xQFoSkYzMWSdiyUTawvu8q98mKQ.1 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file.mpg.2~ETL7xQFoSkYzMWSdiyUTawvu8q98mKQ.2 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file.mpg.2~ETL7xQFoSkYzMWSdiyUTawvu8q98mKQ.3 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file.mpg.2~ETL7xQFoSkYzMWSdiyUTawvu8q98mKQ.4 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.42__multipart_file.mpg.2~ETL7xQFoSkYzMWSdiyUTawvu8q98mKQ.5 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16857955669528843773', time=2024-05-09T22:32:55.747770+0000, chain.objs.size()=11 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.1 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.2 2024-05-09T20:35:49.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.3 2024-05-09T20:35:49.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.4 2024-05-09T20:35:49.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.5 2024-05-09T20:35:49.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.6 2024-05-09T20:35:49.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.747+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.7 2024-05-09T20:35:49.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.751+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.8 2024-05-09T20:35:49.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.751+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.9 2024-05-09T20:35:49.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.751+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.10 2024-05-09T20:35:49.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.751+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~W-4JQ5Tenn0VWNYdc-HsDFMHDQlY8-5.11 2024-05-09T20:35:49.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.759+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.763+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:35:49.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.763+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.763+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.763+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.767+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:35:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.767+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.767+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.767+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.767+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:35:49.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:35:49.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6546353160595275760', time=2024-05-09T22:34:11.904914+0000, chain.objs.size()=11 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.1 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.1_1 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.2 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.2_1 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.3 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.3_1 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.4 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.4_1 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.5 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__shadow_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.5_1 2024-05-09T20:35:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.771+0000 7f0e95298980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.44__multipart_key1.2~Wvtc_5RL3HJc2vLV-ACqivh5mO8nDn0.6 2024-05-09T20:35:49.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.779+0000 7f0e95298980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:35:49.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.783+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:35:49.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.787+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.787+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.787+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.787+0000 7f0e95298980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:35:49.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.791+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.791+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:49.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.791+0000 7f0e95298980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:49.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.791+0000 7f0d497e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:35:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.795+0000 7f0e95298980 20 remove_watcher() i=0 2024-05-09T20:35:49.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.795+0000 7f0e95298980 2 removed watcher, disabling cache 2024-05-09T20:35:49.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.795+0000 7f0e95298980 20 remove_watcher() i=1 2024-05-09T20:35:49.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.799+0000 7f0e95298980 20 remove_watcher() i=2 2024-05-09T20:35:49.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.799+0000 7f0e95298980 20 remove_watcher() i=3 2024-05-09T20:35:49.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.799+0000 7f0e95298980 20 remove_watcher() i=4 2024-05-09T20:35:49.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.803+0000 7f0e95298980 20 remove_watcher() i=5 2024-05-09T20:35:49.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.803+0000 7f0e95298980 20 remove_watcher() i=6 2024-05-09T20:35:49.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.803+0000 7f0e95298980 20 remove_watcher() i=7 2024-05-09T20:35:49.809 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:35:49.876 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:35:49.876 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:35:49.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.923+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.923+0000 7fe52d323980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.923+0000 7fe52d323980 20 realm 2024-05-09T20:35:49.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.923+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.923+0000 7fe52d323980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.923+0000 7fe52d323980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:35:49.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.923+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.927+0000 7fe52d323980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:49.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.927+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.927+0000 7fe52d323980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:35:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.927+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.927+0000 7fe52d323980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:35:49.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.927+0000 7fe52d323980 20 searching for the correct realm 2024-05-09T20:35:49.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:49.931+0000 7fe52ae37640 0 -- 172.21.15.63:0/1548089901 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fe51c0fec20 msgr2=0x7fe51c0fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:35:50.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.147+0000 7fe52ae37640 0 -- 172.21.15.63:0/1548089901 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fe51c0fec20 msgr2=0x7fe51c0fba10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:35:50.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:35:50.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:35:50.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:35:50.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:35:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:35:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:35:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:35:50.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:35:50.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 zone default found 2024-05-09T20:35:50.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 4 Realm: () 2024-05-09T20:35:50.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:35:50.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:35:50.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:35:50.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 zonegroup default 2024-05-09T20:35:50.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.159+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.163+0000 7fe52d323980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.163+0000 7fe52d323980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.163+0000 7fe52d323980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.163+0000 7fe52d323980 20 started sync module instance, tier type = 2024-05-09T20:35:50.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.163+0000 7fe52d323980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:35:50.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.183+0000 7fe52d323980 20 add_watcher() i=0 2024-05-09T20:35:50.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.183+0000 7fe52d323980 20 add_watcher() i=1 2024-05-09T20:35:50.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.183+0000 7fe52d323980 20 add_watcher() i=2 2024-05-09T20:35:50.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.183+0000 7fe52d323980 20 add_watcher() i=3 2024-05-09T20:35:50.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.183+0000 7fe52d323980 20 add_watcher() i=4 2024-05-09T20:35:50.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.183+0000 7fe52d323980 20 add_watcher() i=5 2024-05-09T20:35:50.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.183+0000 7fe52d323980 20 add_watcher() i=6 2024-05-09T20:35:50.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.187+0000 7fe52d323980 20 add_watcher() i=7 2024-05-09T20:35:50.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.187+0000 7fe52d323980 2 all 8 watchers are set, enabling cache 2024-05-09T20:35:50.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.195+0000 7fe3e17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:35:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.195+0000 7fe52d323980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:35:50.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.195+0000 7fe52d323980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:35:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.283+0000 7fe35bfff640 20 reqs_thread_entry: start 2024-05-09T20:35:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.283+0000 7fe52d323980 20 init_complete bucket index max shards: 11 2024-05-09T20:35:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.283+0000 7fe52d323980 20 Filter name: none 2024-05-09T20:35:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.283+0000 7fe359ffb640 20 reqs_thread_entry: start 2024-05-09T20:35:50.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.283+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:35:50.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.287+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.287+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.287+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.291+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:35:50.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.295+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.295+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.295+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.299+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:35:50.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.303+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.303+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.303+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.307+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:35:50.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.311+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.311+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.311+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.315+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:35:50.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.315+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.319+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.319+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.323+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:35:50.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.327+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.327+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.327+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.331+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:35:50.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.331+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.335+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.335+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.335+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:35:50.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.339+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.343+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.336 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.343+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.347+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:35:50.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.351+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.351+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.351+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.355+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:35:50.351 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.359+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.359+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.359+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.363+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:35:50.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.367+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.367+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.371+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.371+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:35:50.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.375+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.379+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.379+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.383+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:35:50.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.383+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.387+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.387+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.391+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:35:50.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.395+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.395+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.395+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.403+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:35:50.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.407+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.407+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.407+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.411+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:35:50.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.415+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.415+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.419+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.419+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:35:50.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.423+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.423+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.427+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.427+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:35:50.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.431+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.435+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.435+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.439+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:35:50.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.443+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.443+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.443+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.447+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:35:50.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.451+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.455+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.455+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.455+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:35:50.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.459+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.463+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.463+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.467+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:35:50.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.471+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.471+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.471+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.475+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:35:50.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.479+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.479+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.475 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.483+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.487+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:35:50.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.487+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.491+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.491+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.495+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:35:50.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.499+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.499+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.499+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.503+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:35:50.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.507+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.507+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.507+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.511+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:35:50.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.515+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.515+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.515+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.519+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:35:50.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.523+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.527+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.527+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.531+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:35:50.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.535+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.535+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.535+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.539+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:35:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.543+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.543+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.543+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.547+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:35:50.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.551+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.551+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.551+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.555+0000 7fe52d323980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:35:50.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.559+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.559+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.559+0000 7fe52d323980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.567+0000 7fe3e17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:35:50.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.571+0000 7fe52d323980 20 remove_watcher() i=0 2024-05-09T20:35:50.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.571+0000 7fe52d323980 2 removed watcher, disabling cache 2024-05-09T20:35:50.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.575+0000 7fe52d323980 20 remove_watcher() i=1 2024-05-09T20:35:50.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.575+0000 7fe52d323980 20 remove_watcher() i=2 2024-05-09T20:35:50.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.579+0000 7fe52d323980 20 remove_watcher() i=3 2024-05-09T20:35:50.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.583+0000 7fe52d323980 20 remove_watcher() i=4 2024-05-09T20:35:50.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.583+0000 7fe52d323980 20 remove_watcher() i=5 2024-05-09T20:35:50.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.587+0000 7fe52d323980 20 remove_watcher() i=6 2024-05-09T20:35:50.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.587+0000 7fe52d323980 20 remove_watcher() i=7 2024-05-09T20:35:50.602 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:35:50.677 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:35:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:35:50.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 20 realm 2024-05-09T20:35:50.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.731+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.735+0000 7fed77429980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.735+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.735+0000 7fed77429980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.735+0000 7fed77429980 20 searching for the correct realm 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.739+0000 7fed7573e640 0 -- 172.21.15.63:0/258078647 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fed68034060 msgr2=0x7fed6805ba70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:35:50.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.743+0000 7fed7473c640 0 -- 172.21.15.63:0/258078647 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x559f9d485360 msgr2=0x7fed6c0f07b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:35:50.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:35:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:35:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:35:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:35:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:35:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:35:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 zone default found 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 4 Realm: () 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 zonegroup default 2024-05-09T20:35:50.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.755+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.759+0000 7fed77429980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.759+0000 7fed77429980 20 rados->read ofs=0 len=0 2024-05-09T20:35:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.759+0000 7fed77429980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.759+0000 7fed77429980 20 started sync module instance, tier type = 2024-05-09T20:35:50.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.759+0000 7fed77429980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:35:50.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 20 add_watcher() i=0 2024-05-09T20:35:50.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 20 add_watcher() i=1 2024-05-09T20:35:50.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 20 add_watcher() i=2 2024-05-09T20:35:50.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 20 add_watcher() i=3 2024-05-09T20:35:50.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 20 add_watcher() i=4 2024-05-09T20:35:50.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 20 add_watcher() i=5 2024-05-09T20:35:50.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 20 add_watcher() i=6 2024-05-09T20:35:50.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 20 add_watcher() i=7 2024-05-09T20:35:50.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.775+0000 7fed77429980 2 all 8 watchers are set, enabling cache 2024-05-09T20:35:50.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.779+0000 7fec2afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:35:50.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.779+0000 7fed77429980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:35:50.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.779+0000 7fed77429980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:35:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.851+0000 7fec29fe3640 20 reqs_thread_entry: start 2024-05-09T20:35:50.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.851+0000 7fed77429980 20 init_complete bucket index max shards: 11 2024-05-09T20:35:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.851+0000 7fed77429980 20 Filter name: none 2024-05-09T20:35:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.851+0000 7feb9b7fe640 20 reqs_thread_entry: start 2024-05-09T20:35:50.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.851+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:35:50.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.855+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.855+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.859+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.859+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:35:50.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.863+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.863+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.863+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.867+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:35:50.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.871+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.871+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.871+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.875+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:35:50.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.875+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.879+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.879+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.879+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:35:50.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.883+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.883+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.887+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.887+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:35:50.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.891+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.895+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.895+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.895+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:35:50.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.899+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.903+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.903+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.898 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.907+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:35:50.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.907+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.907+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.907+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.911+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:35:50.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.915+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.915+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.915+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.919+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:35:50.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.923+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.923+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.916 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.923+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.927+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:35:50.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.927+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.927+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.922 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.931+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.931+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:35:50.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.935+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.935+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.939+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.939+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:35:50.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.943+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.947+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.947+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.951+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:35:50.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.955+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.955+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.949 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.955+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.959+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:35:50.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.963+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.963+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.967+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.967+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:35:50.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.971+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.975+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.975+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.975+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:35:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.979+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.979+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.979+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.983+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:35:50.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.987+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.987+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.991+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.991+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:35:50.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.995+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.995+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.999+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:50.999+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:35:50.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.003+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:50.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.007+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:50.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.007+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.007+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:35:51.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.011+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.015+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.015+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.019+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:35:51.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.023+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.023+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.023+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.027+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:35:51.024 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.031+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.035+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.027 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.035+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.035+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:35:51.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.039+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.043+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.043+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.047+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:35:51.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.051+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.051+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.051+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.055+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:35:51.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.059+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.059+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.059+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.063+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:35:51.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.067+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.071+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.071+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.071+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:35:51.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.075+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.079+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.079+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.083+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:35:51.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.083+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.087+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.087+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.091+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:35:51.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.091+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.095+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.095+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.095+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:35:51.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.099+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.099+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.103+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.103+0000 7fed77429980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:35:51.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.107+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.107+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.111+0000 7fed77429980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.115+0000 7fec2afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:35:51.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.119+0000 7fed77429980 20 remove_watcher() i=0 2024-05-09T20:35:51.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.119+0000 7fed77429980 2 removed watcher, disabling cache 2024-05-09T20:35:51.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.123+0000 7fed77429980 20 remove_watcher() i=1 2024-05-09T20:35:51.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.123+0000 7fed77429980 20 remove_watcher() i=2 2024-05-09T20:35:51.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.127+0000 7fed77429980 20 remove_watcher() i=3 2024-05-09T20:35:51.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.131+0000 7fed77429980 20 remove_watcher() i=4 2024-05-09T20:35:51.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.131+0000 7fed77429980 20 remove_watcher() i=5 2024-05-09T20:35:51.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.135+0000 7fed77429980 20 remove_watcher() i=6 2024-05-09T20:35:51.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.135+0000 7fed77429980 20 remove_watcher() i=7 2024-05-09T20:35:51.149 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:35:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:35:51.232 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:35:51.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:51.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 20 realm 2024-05-09T20:35:51.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:51.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:35:51.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:51.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.287+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.291+0000 7f1f75c47980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:35:51.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.291+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.291+0000 7f1f75c47980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:35:51.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.291+0000 7f1f75c47980 20 searching for the correct realm 2024-05-09T20:35:51.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.295+0000 7f1f7375b640 0 -- 172.21.15.63:0/3765646470 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f1f6c0b8960 msgr2=0x7f1f6c0d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:35:51.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.511+0000 7f1f7375b640 0 -- 172.21.15.63:0/3765646470 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f1f6c0b8960 msgr2=0x7f1f6c0d2430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:35:51.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:35:51.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:35:51.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:35:51.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:35:51.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:35:51.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:35:51.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:51.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:35:51.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.523+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 zone default found 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 4 Realm: () 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 zonegroup default 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:51.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 rados->read ofs=0 len=0 2024-05-09T20:35:51.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:35:51.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 started sync module instance, tier type = 2024-05-09T20:35:51.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.527+0000 7f1f75c47980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:35:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 20 add_watcher() i=0 2024-05-09T20:35:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 20 add_watcher() i=1 2024-05-09T20:35:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 20 add_watcher() i=2 2024-05-09T20:35:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 20 add_watcher() i=3 2024-05-09T20:35:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 20 add_watcher() i=4 2024-05-09T20:35:51.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 20 add_watcher() i=5 2024-05-09T20:35:51.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 20 add_watcher() i=6 2024-05-09T20:35:51.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 20 add_watcher() i=7 2024-05-09T20:35:51.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.551+0000 7f1f75c47980 2 all 8 watchers are set, enabling cache 2024-05-09T20:35:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.555+0000 7f1e29fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:35:51.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.559+0000 7f1f75c47980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:35:51.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.559+0000 7f1f75c47980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:35:51.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.643+0000 7f1e28fe1640 20 reqs_thread_entry: start 2024-05-09T20:35:51.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.643+0000 7f1f75c47980 20 init_complete bucket index max shards: 11 2024-05-09T20:35:51.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.643+0000 7f1f75c47980 20 Filter name: none 2024-05-09T20:35:51.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.643+0000 7f1da27fc640 20 reqs_thread_entry: start 2024-05-09T20:35:51.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.643+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:35:51.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.647+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.647+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.647+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.651+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:35:51.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.655+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.655+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.655+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.659+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:35:51.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.663+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.663+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.663+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.667+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:35:51.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.671+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.671+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.671+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.675+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:35:51.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.679+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.679+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.679+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.683+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:35:51.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.687+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.687+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.687+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.691+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:35:51.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.695+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.695+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.695+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.699+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:35:51.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.703+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.703+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.707+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.707+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:35:51.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.711+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.711+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.705 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.711+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.715+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:35:51.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.719+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.719+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.719+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.723+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:35:51.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.727+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.727+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.731+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.731+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:35:51.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.735+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.735+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.739+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.739+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:35:51.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.743+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.747+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.747+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.751+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:35:51.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.755+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.755+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.755+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.759+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:35:51.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.763+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.763+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.763+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.767+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:35:51.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.771+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.771+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.775+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.775+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:35:51.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.779+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.783+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.783+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.783+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:35:51.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.791+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.791+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.791+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.795+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:35:51.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.799+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.799+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.799+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.803+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:35:51.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.807+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.807+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.811+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.811+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:35:51.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.815+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.815+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.819+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.819+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:35:51.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.823+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.823+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.823+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.827+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:35:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.831+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.831+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.831+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.835+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:35:51.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.839+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.839+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.839+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.839+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:35:51.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.843+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.847+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.847+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.851+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:35:51.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.855+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.855+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.855+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.859+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:35:51.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.863+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.863+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.867+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.867+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:35:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.871+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.871+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.871+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.875+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:35:51.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.879+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.879+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.879+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.879+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:35:51.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.883+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.887+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.887+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.891+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:35:51.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.891+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.895+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.895+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.895+0000 7f1f75c47980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:35:51.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.899+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.899+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:35:51.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.903+0000 7f1f75c47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:35:51.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.907+0000 7f1e29fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:35:51.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.911+0000 7f1f75c47980 20 remove_watcher() i=0 2024-05-09T20:35:51.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.911+0000 7f1f75c47980 2 removed watcher, disabling cache 2024-05-09T20:35:51.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.915+0000 7f1f75c47980 20 remove_watcher() i=1 2024-05-09T20:35:51.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.915+0000 7f1f75c47980 20 remove_watcher() i=2 2024-05-09T20:35:51.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.919+0000 7f1f75c47980 20 remove_watcher() i=3 2024-05-09T20:35:51.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.923+0000 7f1f75c47980 20 remove_watcher() i=4 2024-05-09T20:35:51.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.923+0000 7f1f75c47980 20 remove_watcher() i=5 2024-05-09T20:35:51.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.927+0000 7f1f75c47980 20 remove_watcher() i=6 2024-05-09T20:35:51.921 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:35:51.927+0000 7f1f75c47980 20 remove_watcher() i=7 2024-05-09T20:35:51.941 DEBUG:teuthology.orchestra.run.smithi063:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2024-05-09T20:35:52.665 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :clean 2024-05-09T20:35:53.462 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :compileJava 2024-05-09T20:35:53.462 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :processResources 2024-05-09T20:35:53.462 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :classes 2024-05-09T20:35:53.962 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:53.962 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :compileTestJava 2024-05-09T20:35:53.963 INFO:teuthology.orchestra.run.smithi063.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-05-09T20:35:53.963 INFO:teuthology.orchestra.run.smithi063.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-05-09T20:35:53.964 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:53.964 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :processTestResources NO-SOURCE 2024-05-09T20:35:53.964 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :testClasses 2024-05-09T20:35:55.963 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:55.963 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :test 2024-05-09T20:35:55.963 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:55.963 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests STANDARD_OUT 2024-05-09T20:35:55.964 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09 20:35:55,363 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-05-09T20:35:55.964 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:55.964 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2024-05-09T20:35:55.964 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:55.965 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2024-05-09T20:35:56.362 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:56.362 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2024-05-09T20:35:56.462 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:56.462 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2024-05-09T20:35:57.162 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:57.162 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2024-05-09T20:35:57.262 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:57.262 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2024-05-09T20:35:58.062 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:58.062 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2024-05-09T20:35:58.662 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:58.662 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2024-05-09T20:35:58.762 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:58.763 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2024-05-09T20:35:59.162 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:59.163 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2024-05-09T20:35:59.963 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:59.963 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2024-05-09T20:35:59.963 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:35:59.963 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2024-05-09T20:36:00.162 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:00.162 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2024-05-09T20:36:00.563 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:00.563 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2024-05-09T20:36:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:01.863 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2024-05-09T20:36:02.062 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:02.062 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2024-05-09T20:36:02.863 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:02.863 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2024-05-09T20:36:02.863 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:02.863 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2024-05-09T20:36:03.063 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:03.063 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------- 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------- 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout:BUILD SUCCESSFUL in 11s 2024-05-09T20:36:03.064 INFO:teuthology.orchestra.run.smithi063.stdout:5 actionable tasks: 5 executed 2024-05-09T20:36:03.084 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:36:03.109 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:36:03.110 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:36:03.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.159+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 realm 2024-05-09T20:36:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:36:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:03.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:36:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.163+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.167+0000 7f670ed3b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:36:03.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.167+0000 7f670ed3b980 20 searching for the correct realm 2024-05-09T20:36:03.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.167+0000 7f670c04e640 0 -- 172.21.15.63:0/2650407903 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f67040f9790 msgr2=0x7f67040c4230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:36:03.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.379+0000 7f670c04e640 0 -- 172.21.15.63:0/2650407903 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f670410f420 msgr2=0x7f670410ed80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:36:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:36:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:36:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:36:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:36:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:36:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:36:03.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:36:03.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:36:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 zone default found 2024-05-09T20:36:03.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 4 Realm: () 2024-05-09T20:36:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:36:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:36:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:36:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 zonegroup default 2024-05-09T20:36:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:03.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.603+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.607+0000 7f670ed3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.607+0000 7f670ed3b980 20 rados->read ofs=0 len=0 2024-05-09T20:36:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.607+0000 7f670ed3b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:03.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.607+0000 7f670ed3b980 20 started sync module instance, tier type = 2024-05-09T20:36:03.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.607+0000 7f670ed3b980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:36:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.635+0000 7f670ed3b980 20 add_watcher() i=0 2024-05-09T20:36:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.635+0000 7f670ed3b980 20 add_watcher() i=1 2024-05-09T20:36:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.635+0000 7f670ed3b980 20 add_watcher() i=2 2024-05-09T20:36:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.635+0000 7f670ed3b980 20 add_watcher() i=3 2024-05-09T20:36:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.635+0000 7f670ed3b980 20 add_watcher() i=4 2024-05-09T20:36:03.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.635+0000 7f670ed3b980 20 add_watcher() i=5 2024-05-09T20:36:03.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.635+0000 7f670ed3b980 20 add_watcher() i=6 2024-05-09T20:36:03.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.639+0000 7f670ed3b980 20 add_watcher() i=7 2024-05-09T20:36:03.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.639+0000 7f670ed3b980 2 all 8 watchers are set, enabling cache 2024-05-09T20:36:03.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.643+0000 7f65c37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:36:03.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.643+0000 7f670ed3b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:36:03.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.643+0000 7f670ed3b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:36:03.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.735+0000 7f65c27e4640 20 reqs_thread_entry: start 2024-05-09T20:36:03.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.739+0000 7f670ed3b980 20 init_complete bucket index max shards: 11 2024-05-09T20:36:03.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.739+0000 7f670ed3b980 20 Filter name: none 2024-05-09T20:36:03.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.739+0000 7f65bb7fe640 20 reqs_thread_entry: start 2024-05-09T20:36:03.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.739+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:36:03.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.739+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.743+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.743+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.743+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:36:03.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.747+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.751+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.751+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.751+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:36:03.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.755+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.759+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.759+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.759+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:36:03.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.763+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.763+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.767+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.767+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:36:03.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.771+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.771+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.771+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.775+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:36:03.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.775+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.775+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.779+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.779+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:36:03.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.783+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.783+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.783+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.787+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:36:03.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.791+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.791+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.791+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.795+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:36:03.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.799+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.799+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.799+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.803+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:36:03.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.803+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.807+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.807+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.807+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:36:03.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.811+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.815+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.815+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.815+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:36:03.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.819+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.823+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.823+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.823+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:36:03.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.827+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.831+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.831+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.831+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:36:03.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.835+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.835+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.835+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.839+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:36:03.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.843+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.843+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.843+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.847+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:36:03.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.851+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.851+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.855+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.855+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:36:03.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.859+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.863+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.863+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.863+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:36:03.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.867+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.871+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.871+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.871+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:36:03.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.875+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.879+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.871 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.879+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.879+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:36:03.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.883+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.883+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.887+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.887+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:36:03.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.891+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.891+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.891+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.895+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:36:03.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.899+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.899+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.899+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.903+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:36:03.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.907+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.911+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.911+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.915+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:36:03.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.919+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.919+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.919+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.923+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:36:03.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.927+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.927+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.920 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.927+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.931+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:36:03.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.931+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.935+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.935+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.935+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:36:03.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.939+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.943+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.943+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.947+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:36:03.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.951+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.951+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.955+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.955+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:36:03.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.959+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.963+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.963+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.963+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:36:03.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.967+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.971+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.971+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.975+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:36:03.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.979+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.979+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.979+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.983+0000 7f670ed3b980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:36:03.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.983+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.987+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:03.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.987+0000 7f670ed3b980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:03.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.991+0000 7f65c37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:36:03.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.999+0000 7f670ed3b980 20 remove_watcher() i=0 2024-05-09T20:36:03.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.999+0000 7f670ed3b980 2 removed watcher, disabling cache 2024-05-09T20:36:03.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:03.999+0000 7f670ed3b980 20 remove_watcher() i=1 2024-05-09T20:36:03.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.003+0000 7f670ed3b980 20 remove_watcher() i=2 2024-05-09T20:36:03.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.007+0000 7f670ed3b980 20 remove_watcher() i=3 2024-05-09T20:36:04.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.007+0000 7f670ed3b980 20 remove_watcher() i=4 2024-05-09T20:36:04.003 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.011+0000 7f670ed3b980 20 remove_watcher() i=5 2024-05-09T20:36:04.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.011+0000 7f670ed3b980 20 remove_watcher() i=6 2024-05-09T20:36:04.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.011+0000 7f670ed3b980 20 remove_watcher() i=7 2024-05-09T20:36:04.026 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:36:04.098 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:36:04.098 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:36:04.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 20 realm 2024-05-09T20:36:04.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:36:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.151+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.155+0000 7fdca1d47980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:36:04.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.155+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.155+0000 7fdca1d47980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:36:04.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.155+0000 7fdca1d47980 20 searching for the correct realm 2024-05-09T20:36:04.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.159+0000 7fdca005c640 0 -- 172.21.15.63:0/1138403165 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fdc94054480 msgr2=0x7fdc940568f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:36:04.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.371+0000 7fdc9f05a640 0 -- 172.21.15.63:0/1138403165 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fdc9005de70 msgr2=0x7fdc9005cad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:36:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:36:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:36:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:36:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:36:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:36:04.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:36:04.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.387+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:36:04.383 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 zone default found 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 4 Realm: () 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 zonegroup default 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 started sync module instance, tier type = 2024-05-09T20:36:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.391+0000 7fdca1d47980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:36:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.419+0000 7fdca1d47980 20 add_watcher() i=0 2024-05-09T20:36:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.419+0000 7fdca1d47980 20 add_watcher() i=1 2024-05-09T20:36:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.419+0000 7fdca1d47980 20 add_watcher() i=2 2024-05-09T20:36:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.419+0000 7fdca1d47980 20 add_watcher() i=3 2024-05-09T20:36:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.419+0000 7fdca1d47980 20 add_watcher() i=4 2024-05-09T20:36:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.419+0000 7fdca1d47980 20 add_watcher() i=5 2024-05-09T20:36:04.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.419+0000 7fdca1d47980 20 add_watcher() i=6 2024-05-09T20:36:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.423+0000 7fdca1d47980 20 add_watcher() i=7 2024-05-09T20:36:04.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.423+0000 7fdca1d47980 2 all 8 watchers are set, enabling cache 2024-05-09T20:36:04.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.427+0000 7fdb55fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:36:04.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.427+0000 7fdca1d47980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:36:04.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.427+0000 7fdca1d47980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:36:04.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.523+0000 7fdb54fe1640 20 reqs_thread_entry: start 2024-05-09T20:36:04.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.523+0000 7fdca1d47980 20 init_complete bucket index max shards: 11 2024-05-09T20:36:04.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.523+0000 7fdca1d47980 20 Filter name: none 2024-05-09T20:36:04.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.523+0000 7fdace7fc640 20 reqs_thread_entry: start 2024-05-09T20:36:04.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.523+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:36:04.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.527+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.527+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.527+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.531+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:36:04.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.531+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.535+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.535+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.539+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:36:04.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.539+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.543+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.543+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.543+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:36:04.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.547+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.547+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.547+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.551+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:36:04.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.555+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.555+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.559+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.559+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:36:04.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.563+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.563+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.567+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.567+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:36:04.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.571+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.571+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.571+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.575+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:36:04.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.579+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.579+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.572 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.579+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.583+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:36:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.583+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.587+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.587+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.587+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:36:04.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.591+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.591+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.586 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.591+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.595+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:36:04.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.599+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.599+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.603+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.603+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:36:04.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.607+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.602 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.607+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.603 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.611+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.611+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:36:04.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.615+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.615+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.615+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.619+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:36:04.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.623+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.623+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.623+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.627+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:36:04.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.631+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.631+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.631+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.635+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:36:04.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.635+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.639+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.639+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.643+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:36:04.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.647+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.641 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.647+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.647+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.651+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:36:04.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.655+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.655+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.655+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.659+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:36:04.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.663+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.663+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.667+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.667+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:36:04.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.671+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.671+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.671+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.675+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:36:04.672 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.679+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.679+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.679+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.683+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:36:04.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.687+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.687+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.687+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.691+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:36:04.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.695+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.695+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.699+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.703+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:36:04.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.703+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.703+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.707+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.707+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:36:04.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.711+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.711+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.715+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.715+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:36:04.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.719+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.723+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.723+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.723+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:36:04.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.727+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.727+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.731+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.731+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:36:04.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.735+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.735+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.739+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.739+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:36:04.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.743+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.743+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.747+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.747+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:36:04.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.751+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.751+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.755+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.755+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:36:04.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.759+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.759+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.759+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.763+0000 7fdca1d47980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:36:04.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.767+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.767+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:04.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.767+0000 7fdca1d47980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:04.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.775+0000 7fdb55fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:36:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.779+0000 7fdca1d47980 20 remove_watcher() i=0 2024-05-09T20:36:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.779+0000 7fdca1d47980 2 removed watcher, disabling cache 2024-05-09T20:36:04.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.783+0000 7fdca1d47980 20 remove_watcher() i=1 2024-05-09T20:36:04.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.787+0000 7fdca1d47980 20 remove_watcher() i=2 2024-05-09T20:36:04.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.791+0000 7fdca1d47980 20 remove_watcher() i=3 2024-05-09T20:36:04.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.791+0000 7fdca1d47980 20 remove_watcher() i=4 2024-05-09T20:36:04.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.795+0000 7fdca1d47980 20 remove_watcher() i=5 2024-05-09T20:36:04.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.795+0000 7fdca1d47980 20 remove_watcher() i=6 2024-05-09T20:36:04.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.799+0000 7fdca1d47980 20 remove_watcher() i=7 2024-05-09T20:36:04.814 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:36:04.890 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:36:04.890 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:36:04.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 20 realm 2024-05-09T20:36:04.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:36:04.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:04.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.943+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.947+0000 7efc27f5a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:36:04.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.947+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:04.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.947+0000 7efc27f5a980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:36:04.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.947+0000 7efc27f5a980 20 searching for the correct realm 2024-05-09T20:36:04.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:04.951+0000 7efc25a6e640 0 -- 172.21.15.63:0/575798966 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7efc2005e440 msgr2=0x7efc20076130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:36:05.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.167+0000 7efc25a6e640 0 -- 172.21.15.63:0/575798966 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7efc2005e440 msgr2=0x7efc20076130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:36:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:36:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:36:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:36:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:36:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:36:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:36:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:36:05.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:36:05.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 zone default found 2024-05-09T20:36:05.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 4 Realm: () 2024-05-09T20:36:05.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:36:05.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:36:05.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:36:05.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 zonegroup default 2024-05-09T20:36:05.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.179+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.183+0000 7efc27f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.183+0000 7efc27f5a980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.183+0000 7efc27f5a980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.183+0000 7efc27f5a980 20 started sync module instance, tier type = 2024-05-09T20:36:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.183+0000 7efc27f5a980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:36:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.207+0000 7efc27f5a980 20 add_watcher() i=0 2024-05-09T20:36:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.207+0000 7efc27f5a980 20 add_watcher() i=1 2024-05-09T20:36:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.207+0000 7efc27f5a980 20 add_watcher() i=2 2024-05-09T20:36:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.207+0000 7efc27f5a980 20 add_watcher() i=3 2024-05-09T20:36:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.207+0000 7efc27f5a980 20 add_watcher() i=4 2024-05-09T20:36:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.207+0000 7efc27f5a980 20 add_watcher() i=5 2024-05-09T20:36:05.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.207+0000 7efc27f5a980 20 add_watcher() i=6 2024-05-09T20:36:05.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.211+0000 7efc27f5a980 20 add_watcher() i=7 2024-05-09T20:36:05.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.211+0000 7efc27f5a980 2 all 8 watchers are set, enabling cache 2024-05-09T20:36:05.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.215+0000 7efad77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:36:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.215+0000 7efc27f5a980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:36:05.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.215+0000 7efc27f5a980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:36:05.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.303+0000 7efad67fc640 20 reqs_thread_entry: start 2024-05-09T20:36:05.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.303+0000 7efc27f5a980 20 init_complete bucket index max shards: 11 2024-05-09T20:36:05.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.303+0000 7efc27f5a980 20 Filter name: none 2024-05-09T20:36:05.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.303+0000 7efa47fff640 20 reqs_thread_entry: start 2024-05-09T20:36:05.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.303+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:36:05.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.307+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.307+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.307+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.311+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:36:05.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.315+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.315+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.315+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.319+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:36:05.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.323+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.323+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.323+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.327+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:36:05.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.327+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.331+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.331+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.331+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:36:05.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.335+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.335+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.335+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.339+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:36:05.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.343+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.343+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.347+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.347+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:36:05.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.351+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.351+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.351+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.355+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:36:05.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.359+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.363+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.363+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.367+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:36:05.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.367+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.367+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.371+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.371+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:36:05.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.375+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.375+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.375+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.375+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:36:05.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.379+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.379+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.383+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.383+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:36:05.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.387+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.387+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.387+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.391+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:36:05.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.391+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.395+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.395+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.399+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:36:05.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.399+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.399+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.403+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.403+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:36:05.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.407+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.411+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.411+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.415+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:36:05.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.415+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.415+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.419+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.419+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:36:05.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.423+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.423+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.423+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.427+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:36:05.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.431+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.431+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.431+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.435+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:36:05.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.439+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.439+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.443+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.443+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:36:05.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.447+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.451+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.451+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.455+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:36:05.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.459+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.463+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.463+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.463+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:36:05.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.467+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.467+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.467+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.471+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:36:05.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.475+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.475+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.475+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.479+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:36:05.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.483+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.483+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.483+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.487+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:36:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.491+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.491+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.491+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.491+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:36:05.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.495+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.499+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.499+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.503+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:36:05.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.503+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.507+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.507+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.511+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:36:05.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.515+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.515+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.515+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.519+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:36:05.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.523+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.523+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.518 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.523+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.527+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:36:05.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.531+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.531+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.531+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.535+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:36:05.530 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.535+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.539+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.539+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.539+0000 7efc27f5a980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:36:05.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.543+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.547+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:05.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.547+0000 7efc27f5a980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:05.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.551+0000 7efad77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:36:05.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.555+0000 7efc27f5a980 20 remove_watcher() i=0 2024-05-09T20:36:05.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.555+0000 7efc27f5a980 2 removed watcher, disabling cache 2024-05-09T20:36:05.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.559+0000 7efc27f5a980 20 remove_watcher() i=1 2024-05-09T20:36:05.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.559+0000 7efc27f5a980 20 remove_watcher() i=2 2024-05-09T20:36:05.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.563+0000 7efc27f5a980 20 remove_watcher() i=3 2024-05-09T20:36:05.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.563+0000 7efc27f5a980 20 remove_watcher() i=4 2024-05-09T20:36:05.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.567+0000 7efc27f5a980 20 remove_watcher() i=5 2024-05-09T20:36:05.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.567+0000 7efc27f5a980 20 remove_watcher() i=6 2024-05-09T20:36:05.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.571+0000 7efc27f5a980 20 remove_watcher() i=7 2024-05-09T20:36:05.582 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:36:05.658 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:36:05.658 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:36:05.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 20 realm 2024-05-09T20:36:05.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:36:05.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.706 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.711+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.715+0000 7fafc91f2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:36:05.708 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.715+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.715+0000 7fafc91f2980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:36:05.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.715+0000 7fafc91f2980 20 searching for the correct realm 2024-05-09T20:36:05.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.719+0000 7fafc7507640 0 -- 172.21.15.63:0/4188545183 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fafbc080040 msgr2=0x7fafbc056350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:36:05.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.931+0000 7fafc6505640 0 -- 172.21.15.63:0/4188545183 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fafb8008050 msgr2=0x7fafb8027920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:36:05.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:36:05.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:36:05.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:36:05.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:36:05.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:36:05.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:36:05.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:36:05.945 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 zone default found 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 4 Realm: () 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 zonegroup default 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.951+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.955+0000 7fafc91f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.955+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.955+0000 7fafc91f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.955+0000 7fafc91f2980 20 rados->read ofs=0 len=0 2024-05-09T20:36:05.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.955+0000 7fafc91f2980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:36:05.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.955+0000 7fafc91f2980 20 started sync module instance, tier type = 2024-05-09T20:36:05.948 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.955+0000 7fafc91f2980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:36:05.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.979+0000 7fafc91f2980 20 add_watcher() i=0 2024-05-09T20:36:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.979+0000 7fafc91f2980 20 add_watcher() i=1 2024-05-09T20:36:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.979+0000 7fafc91f2980 20 add_watcher() i=2 2024-05-09T20:36:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.979+0000 7fafc91f2980 20 add_watcher() i=3 2024-05-09T20:36:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.979+0000 7fafc91f2980 20 add_watcher() i=4 2024-05-09T20:36:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.979+0000 7fafc91f2980 20 add_watcher() i=5 2024-05-09T20:36:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.979+0000 7fafc91f2980 20 add_watcher() i=6 2024-05-09T20:36:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.983+0000 7fafc91f2980 20 add_watcher() i=7 2024-05-09T20:36:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.983+0000 7fafc91f2980 2 all 8 watchers are set, enabling cache 2024-05-09T20:36:05.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.983+0000 7fae7d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:36:05.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.987+0000 7fafc91f2980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:36:05.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:05.987+0000 7fafc91f2980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:36:06.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.079+0000 7fadf7fff640 20 reqs_thread_entry: start 2024-05-09T20:36:06.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.079+0000 7fafc91f2980 20 init_complete bucket index max shards: 11 2024-05-09T20:36:06.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.079+0000 7fafc91f2980 20 Filter name: none 2024-05-09T20:36:06.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.079+0000 7fadf5ffb640 20 reqs_thread_entry: start 2024-05-09T20:36:06.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.079+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:36:06.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.083+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.076 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.083+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.083+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.087+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:36:06.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.091+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.091+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.091+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.095+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:36:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.099+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.099+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.103+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.103+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:36:06.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.107+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.103 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.111+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.111+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.115+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:36:06.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.115+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.119+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.119+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.119+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:36:06.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.123+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.127+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.127+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.131+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:36:06.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.131+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.135+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.135+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.135+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:36:06.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.139+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.143+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.143+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.147+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:36:06.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.151+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.151+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.151+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.155+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:36:06.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.159+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.159+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.163+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.163+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:36:06.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.167+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.167+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.171+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.171+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:36:06.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.175+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.179+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.179+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.179+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:36:06.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.183+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.187+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.187+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.191+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:36:06.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.195+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.195+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.199+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.199+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:36:06.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.203+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.207+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.207+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.211+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:36:06.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.215+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.215+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.215+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.219+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:36:06.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.223+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.223+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.223+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.227+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:36:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.231+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.231+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.235+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.235+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:36:06.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.239+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.243+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.243+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.243+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:36:06.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.247+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.251+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.251+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.255+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:36:06.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.259+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.259+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.263+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.263+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:36:06.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.267+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.271+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.271+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.275+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:36:06.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.279+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.279+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.279+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.283+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:36:06.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.287+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.287+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.287+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.291+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:36:06.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.291+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.295+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.295+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.299+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:36:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.299+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.303+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.303+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.307+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:36:06.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.307+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.311+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.304 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.311+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.315+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:36:06.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.319+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.319+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.319+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.323+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:36:06.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.327+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.327+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.331+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.331+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:36:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.335+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.339+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.339+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.339+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:36:06.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.343+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.347+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.347+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.347+0000 7fafc91f2980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:36:06.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.351+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.355+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:36:06.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.355+0000 7fafc91f2980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:36:06.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.359+0000 7fae7d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:36:06.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.363+0000 7fafc91f2980 20 remove_watcher() i=0 2024-05-09T20:36:06.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.363+0000 7fafc91f2980 2 removed watcher, disabling cache 2024-05-09T20:36:06.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.367+0000 7fafc91f2980 20 remove_watcher() i=1 2024-05-09T20:36:06.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.371+0000 7fafc91f2980 20 remove_watcher() i=2 2024-05-09T20:36:06.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.371+0000 7fafc91f2980 20 remove_watcher() i=3 2024-05-09T20:36:06.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.375+0000 7fafc91f2980 20 remove_watcher() i=4 2024-05-09T20:36:06.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.375+0000 7fafc91f2980 20 remove_watcher() i=5 2024-05-09T20:36:06.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.379+0000 7fafc91f2980 20 remove_watcher() i=6 2024-05-09T20:36:06.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:36:06.379+0000 7fafc91f2980 20 remove_watcher() i=7 2024-05-09T20:36:06.394 DEBUG:teuthology.orchestra.run.smithi063:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2024-05-09T20:36:07.126 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :clean 2024-05-09T20:36:07.824 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :compileJava 2024-05-09T20:36:07.824 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :processResources 2024-05-09T20:36:07.824 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :classes 2024-05-09T20:36:08.327 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:08.327 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :compileTestJava 2024-05-09T20:36:08.328 INFO:teuthology.orchestra.run.smithi063.stderr:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2024-05-09T20:36:08.328 INFO:teuthology.orchestra.run.smithi063.stderr:Note: Recompile with -Xlint:deprecation for details. 2024-05-09T20:36:08.329 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:08.329 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :processTestResources NO-SOURCE 2024-05-09T20:36:08.329 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :testClasses 2024-05-09T20:36:10.324 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:10.324 INFO:teuthology.orchestra.run.smithi063.stdout:> Task :test 2024-05-09T20:36:10.324 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:10.324 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests STANDARD_OUT 2024-05-09T20:36:10.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09 20:36:09,785 [Test worker] WARN com.amazonaws.util.Base64 - JAXB is unavailable. Will fallback to SDK implementation which may be less performant 2024-05-09T20:36:10.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:10.325 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2024-05-09T20:36:10.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:10.325 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2024-05-09T20:36:11.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:11.524 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2024-05-09T20:36:11.724 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:11.724 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2024-05-09T20:36:12.424 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:12.424 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2024-05-09T20:36:12.624 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:12.624 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2024-05-09T20:36:13.424 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:13.424 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2024-05-09T20:36:26.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:26.525 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2024-05-09T20:36:27.224 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:27.224 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2024-05-09T20:36:27.424 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:27.424 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2024-05-09T20:36:28.024 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:28.024 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2024-05-09T20:36:28.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:28.524 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2024-05-09T20:36:29.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:29.525 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2024-05-09T20:36:29.525 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:29.525 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2024-05-09T20:36:29.724 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:29.724 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2024-05-09T20:36:29.824 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:29.824 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2024-05-09T20:36:30.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:30.524 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2024-05-09T20:36:32.924 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:32.924 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2024-05-09T20:36:33.724 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:33.724 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2024-05-09T20:36:33.924 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:33.924 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2024-05-09T20:36:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:34.624 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2024-05-09T20:36:34.824 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:34.824 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2024-05-09T20:36:35.624 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:35.624 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2024-05-09T20:36:35.827 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:35.827 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2024-05-09T20:36:36.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:36.524 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2024-05-09T20:36:36.724 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:36.724 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2024-05-09T20:36:37.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:37.524 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2024-05-09T20:36:37.724 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:37.724 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2024-05-09T20:36:38.424 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:38.425 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2024-05-09T20:36:38.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:38.524 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2024-05-09T20:36:39.224 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:39.224 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2024-05-09T20:36:39.724 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:39.724 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2024-05-09T20:36:40.324 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:40.325 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2024-05-09T20:36:53.725 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:53.725 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2024-05-09T20:36:55.224 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:36:55.225 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2024-05-09T20:37:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:50.726 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2024-05-09T20:37:52.126 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:52.126 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2024-05-09T20:37:52.226 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:52.226 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2024-05-09T20:37:52.926 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:52.926 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2024-05-09T20:37:53.225 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:53.226 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2024-05-09T20:37:54.426 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:54.426 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2024-05-09T20:37:57.826 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:57.826 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2024-05-09T20:37:58.626 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:58.626 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2024-05-09T20:37:58.726 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:58.726 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2024-05-09T20:37:58.826 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:37:58.826 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2024-05-09T20:38:03.526 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:03.526 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2024-05-09T20:38:04.926 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:04.926 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests STANDARD_OUT 2024-05-09T20:38:04.927 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09 20:38:04,844 [s3-transfer-manager-worker-10] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-05-09T20:38:04.927 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09 20:38:04,845 [s3-transfer-manager-worker-9] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-05-09T20:38:04.927 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09 20:38:04,856 [s3-transfer-manager-worker-2] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-05-09T20:38:04.927 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09 20:38:04,857 [s3-transfer-manager-worker-1] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2024-05-09T20:38:05.526 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:05.526 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2024-05-09T20:38:12.226 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2024-05-09T20:38:13.226 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:13.226 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2024-05-09T20:38:17.326 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:17.326 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2024-05-09T20:38:18.026 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:18.026 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2024-05-09T20:38:37.127 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:37.127 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2024-05-09T20:38:37.826 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:37.827 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2024-05-09T20:38:42.527 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:42.527 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2024-05-09T20:38:43.226 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:43.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2024-05-09T20:38:56.527 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:56.527 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2024-05-09T20:38:57.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:38:57.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2024-05-09T20:39:01.427 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:01.427 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2024-05-09T20:39:17.327 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:17.327 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2024-05-09T20:39:17.527 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:17.527 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2024-05-09T20:39:17.827 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:17.827 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2024-05-09T20:39:18.327 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:18.327 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2024-05-09T20:39:22.527 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:22.528 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2024-05-09T20:39:22.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:22.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2024-05-09T20:39:23.428 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:23.428 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2024-05-09T20:39:23.627 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:23.627 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2024-05-09T20:39:24.327 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:24.327 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2024-05-09T20:39:24.527 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:24.527 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2024-05-09T20:39:25.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:25.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2024-05-09T20:39:25.427 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:25.427 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2024-05-09T20:39:26.127 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:26.127 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2024-05-09T20:39:26.427 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:26.427 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2024-05-09T20:39:27.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:27.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2024-05-09T20:39:27.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:27.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2024-05-09T20:39:28.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:28.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2024-05-09T20:39:29.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:29.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2024-05-09T20:39:30.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:30.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2024-05-09T20:39:30.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2024-05-09T20:39:31.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:31.728 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2024-05-09T20:39:32.228 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:32.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2024-05-09T20:39:33.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:33.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2024-05-09T20:39:33.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:33.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2024-05-09T20:39:34.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:34.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2024-05-09T20:39:35.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:35.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2024-05-09T20:39:36.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:36.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2024-05-09T20:39:36.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:36.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2024-05-09T20:39:37.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:37.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2024-05-09T20:39:38.228 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:38.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2024-05-09T20:39:39.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:39.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2024-05-09T20:39:39.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:39.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2024-05-09T20:39:40.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:40.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2024-05-09T20:39:41.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:41.227 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2024-05-09T20:39:42.228 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:42.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2024-05-09T20:39:42.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:42.728 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2024-05-09T20:39:43.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:43.727 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2024-05-09T20:39:44.227 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:44.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2024-05-09T20:39:45.027 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:45.027 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2024-05-09T20:39:45.527 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:45.528 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2024-05-09T20:39:46.428 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:46.428 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2024-05-09T20:39:47.027 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:47.028 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2024-05-09T20:39:48.027 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:48.028 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2024-05-09T20:39:48.528 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:48.528 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2024-05-09T20:39:49.527 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:49.527 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2024-05-09T20:39:49.928 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:49.928 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2024-05-09T20:39:50.928 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:50.928 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2024-05-09T20:39:51.427 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:51.427 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2024-05-09T20:39:52.428 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:52.428 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2024-05-09T20:39:53.028 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:53.028 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2024-05-09T20:39:54.027 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:54.028 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2024-05-09T20:39:54.427 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:54.428 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2024-05-09T20:39:55.328 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:55.328 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2024-05-09T20:39:55.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:55.728 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2024-05-09T20:39:56.727 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:56.728 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2024-05-09T20:39:57.128 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:57.128 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2024-05-09T20:39:58.128 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:58.128 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2024-05-09T20:39:58.528 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:58.528 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2024-05-09T20:39:59.428 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:59.428 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2024-05-09T20:39:59.628 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:39:59.628 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2024-05-09T20:40:00.328 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:00.328 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2024-05-09T20:40:00.528 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:00.528 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2024-05-09T20:40:01.228 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:01.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2024-05-09T20:40:01.428 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:01.428 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2024-05-09T20:40:02.228 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:02.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2024-05-09T20:40:02.427 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:02.428 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2024-05-09T20:40:03.228 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:03.228 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2024-05-09T20:40:03.828 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:03.828 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2024-05-09T20:40:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:04.528 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2024-05-09T20:40:04.528 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:04.528 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2024-05-09T20:40:04.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:04.728 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2024-05-09T20:40:04.927 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:04.928 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2024-05-09T20:40:05.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:05.728 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2024-05-09T20:40:06.028 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:06.028 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2024-05-09T20:40:06.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:06.728 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2024-05-09T20:40:07.128 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:07.128 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2024-05-09T20:40:07.828 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:07.828 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2024-05-09T20:40:08.028 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:08.028 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2024-05-09T20:40:08.728 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:08.728 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2024-05-09T20:40:22.028 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:22.029 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2024-05-09T20:40:22.828 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:22.828 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testUploadFileList STARTED 2024-05-09T20:40:27.028 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:27.028 INFO:teuthology.orchestra.run.smithi063.stdout:suite > Object tests > ObjectTest.testUploadFileList PASSED 2024-05-09T20:40:27.925 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:27.925 INFO:teuthology.orchestra.run.smithi063.stdout:---------------------------------------------------------------------- 2024-05-09T20:40:27.925 INFO:teuthology.orchestra.run.smithi063.stdout:| Results: SUCCESS (70 tests, 70 successes, 0 failures, 0 skipped) | 2024-05-09T20:40:27.926 INFO:teuthology.orchestra.run.smithi063.stdout:---------------------------------------------------------------------- 2024-05-09T20:40:27.926 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:27.926 INFO:teuthology.orchestra.run.smithi063.stdout:Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. 2024-05-09T20:40:27.926 INFO:teuthology.orchestra.run.smithi063.stdout:Use '--warning-mode all' to show the individual deprecation warnings. 2024-05-09T20:40:27.926 INFO:teuthology.orchestra.run.smithi063.stdout:See https://docs.gradle.org/6.0.1/userguide/command_line_interface.html#sec:command_line_warnings 2024-05-09T20:40:27.926 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:27.926 INFO:teuthology.orchestra.run.smithi063.stdout:BUILD SUCCESSFUL in 4m 21s 2024-05-09T20:40:27.926 INFO:teuthology.orchestra.run.smithi063.stdout:5 actionable tasks: 5 executed 2024-05-09T20:40:27.934 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:40:27.965 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:27.965 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:28.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.012+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 realm 2024-05-09T20:40:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:28.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.016+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.020+0000 7ff2c35ad980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:28.016 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.020+0000 7ff2c35ad980 20 searching for the correct realm 2024-05-09T20:40:28.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.020+0000 7ff2c18c2640 0 -- 172.21.15.63:0/3428526064 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7ff2b8049070 msgr2=0x7ff2b803f910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:28.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.028+0000 7ff2c08c0640 0 -- 172.21.15.63:0/3428526064 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7ff2b40da070 msgr2=0x7ff2b40f6550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:28.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:28.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 zone default found 2024-05-09T20:40:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 4 Realm: () 2024-05-09T20:40:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 zonegroup default 2024-05-09T20:40:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 started sync module instance, tier type = 2024-05-09T20:40:28.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.040+0000 7ff2c35ad980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:28.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 20 add_watcher() i=0 2024-05-09T20:40:28.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 20 add_watcher() i=1 2024-05-09T20:40:28.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 20 add_watcher() i=2 2024-05-09T20:40:28.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 20 add_watcher() i=3 2024-05-09T20:40:28.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 20 add_watcher() i=4 2024-05-09T20:40:28.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 20 add_watcher() i=5 2024-05-09T20:40:28.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 20 add_watcher() i=6 2024-05-09T20:40:28.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 20 add_watcher() i=7 2024-05-09T20:40:28.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.056+0000 7ff2c35ad980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:28.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.060+0000 7ff1777e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.060+0000 7ff2c35ad980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:28.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.060+0000 7ff2c35ad980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:28.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.140+0000 7ff1767e4640 20 reqs_thread_entry: start 2024-05-09T20:40:28.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.140+0000 7ff2c35ad980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:28.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.140+0000 7ff2c35ad980 20 Filter name: none 2024-05-09T20:40:28.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.140+0000 7ff0e7fff640 20 reqs_thread_entry: start 2024-05-09T20:40:28.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.140+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:40:28.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.140+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.144+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.144+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.148+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:40:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.148+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.148+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.148+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.152+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:40:28.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.152+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.152+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.152+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.156+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:40:28.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:40:28.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6431522810940602462', time=2024-05-09T22:40:27.225671+0000, chain.objs.size()=5 2024-05-09T20:40:28.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.133__multipart_key1/data/file.mpg.2~r8KeM9kl2_RMVPCjswYyw5ii3m7UzDZ.1 2024-05-09T20:40:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.133__multipart_key1/data/file.mpg.2~r8KeM9kl2_RMVPCjswYyw5ii3m7UzDZ.2 2024-05-09T20:40:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.133__multipart_key1/data/file.mpg.2~r8KeM9kl2_RMVPCjswYyw5ii3m7UzDZ.3 2024-05-09T20:40:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.133__multipart_key1/data/file.mpg.2~r8KeM9kl2_RMVPCjswYyw5ii3m7UzDZ.4 2024-05-09T20:40:28.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.160+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.133__multipart_key1/data/file.mpg.2~r8KeM9kl2_RMVPCjswYyw5ii3m7UzDZ.5 2024-05-09T20:40:28.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.168+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.172+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:40:28.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:40:28.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7650657937708185790', time=2024-05-09T22:37:40.292957+0000, chain.objs.size()=5 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~PYg2Qot5PiE0bZiTvTlasUpp7ouPPM3.1 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~PYg2Qot5PiE0bZiTvTlasUpp7ouPPM3.2 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~PYg2Qot5PiE0bZiTvTlasUpp7ouPPM3.3 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~PYg2Qot5PiE0bZiTvTlasUpp7ouPPM3.4 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~PYg2Qot5PiE0bZiTvTlasUpp7ouPPM3.5 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14883398264059017953', time=2024-05-09T22:40:22.241671+0000, chain.objs.size()=5 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.132__multipart_key1.2~ksB67Bcm26qLxjyp_3ULynXz1IllK0q.1 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.132__multipart_key1.2~ksB67Bcm26qLxjyp_3ULynXz1IllK0q.2 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.132__multipart_key1.2~ksB67Bcm26qLxjyp_3ULynXz1IllK0q.3 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.132__multipart_key1.2~ksB67Bcm26qLxjyp_3ULynXz1IllK0q.4 2024-05-09T20:40:28.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.176+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.132__multipart_key1.2~ksB67Bcm26qLxjyp_3ULynXz1IllK0q.5 2024-05-09T20:40:28.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.192+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.196+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:40:28.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.196+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.196+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.196+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.200+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.204+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.204+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.204+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.204+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.208+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.13800945410019443891', time=2024-05-09T22:38:28.108603+0000, chain.objs.size()=5 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~sNgqzEABakfjdP2luu9GwD-pZlOyPFs.1 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~sNgqzEABakfjdP2luu9GwD-pZlOyPFs.2 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~sNgqzEABakfjdP2luu9GwD-pZlOyPFs.3 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~sNgqzEABakfjdP2luu9GwD-pZlOyPFs.4 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~sNgqzEABakfjdP2luu9GwD-pZlOyPFs.5 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.107097494730528597', time=2024-05-09T22:38:37.297935+0000, chain.objs.size()=5 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~NDE4--BQWENggGoKjfyjDxN9Avuo6wB.1 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__shadow_key1.2~NDE4--BQWENggGoKjfyjDxN9Avuo6wB.1_1 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~NDE4--BQWENggGoKjfyjDxN9Avuo6wB.2 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__shadow_key1.2~NDE4--BQWENggGoKjfyjDxN9Avuo6wB.2_1 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.212+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~NDE4--BQWENggGoKjfyjDxN9Avuo6wB.3 2024-05-09T20:40:28.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.224+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.232+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12671343786243797199', time=2024-05-09T22:38:36.953245+0000, chain.objs.size()=5 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~I3t2qCe_0zmU4sErkxqzjDnOUFaVX2t.1 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__shadow_key1.2~I3t2qCe_0zmU4sErkxqzjDnOUFaVX2t.1_1 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~I3t2qCe_0zmU4sErkxqzjDnOUFaVX2t.2 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__shadow_key1.2~I3t2qCe_0zmU4sErkxqzjDnOUFaVX2t.2_1 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.236+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~I3t2qCe_0zmU4sErkxqzjDnOUFaVX2t.3 2024-05-09T20:40:28.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.240+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.248+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:40:28.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.252+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.252+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.252+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:40:28.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.7366993941423898862', time=2024-05-09T22:38:12.395091+0000, chain.objs.size()=5 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file.mpg.2~Jlx0SnDoWHbdypbo8Unvr2hUXgdFoFv.1 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file.mpg.2~Jlx0SnDoWHbdypbo8Unvr2hUXgdFoFv.2 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file.mpg.2~Jlx0SnDoWHbdypbo8Unvr2hUXgdFoFv.3 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file.mpg.2~Jlx0SnDoWHbdypbo8Unvr2hUXgdFoFv.4 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file.mpg.2~Jlx0SnDoWHbdypbo8Unvr2hUXgdFoFv.5 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18377358419969156713', time=2024-05-09T22:38:24.777110+0000, chain.objs.size()=5 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~9Qs9uowxZwh3xNcD6uEyEEooTVfBle5.1 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~9Qs9uowxZwh3xNcD6uEyEEooTVfBle5.2 2024-05-09T20:40:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~9Qs9uowxZwh3xNcD6uEyEEooTVfBle5.3 2024-05-09T20:40:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~9Qs9uowxZwh3xNcD6uEyEEooTVfBle5.4 2024-05-09T20:40:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.256+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~9Qs9uowxZwh3xNcD6uEyEEooTVfBle5.5 2024-05-09T20:40:28.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.268+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.276+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:40:28.274 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.276+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.280+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.280+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.280+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:40:28.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:40:28.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.11912737002742121124', time=2024-05-09T22:38:42.675913+0000, chain.objs.size()=5 2024-05-09T20:40:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.92__multipart_key1.2~yA34xoRwHPAIX1AeiluS_V40_u9NCNw.1 2024-05-09T20:40:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.92__multipart_key1.2~yA34xoRwHPAIX1AeiluS_V40_u9NCNw.2 2024-05-09T20:40:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.92__multipart_key1.2~yA34xoRwHPAIX1AeiluS_V40_u9NCNw.3 2024-05-09T20:40:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.92__multipart_key1.2~yA34xoRwHPAIX1AeiluS_V40_u9NCNw.4 2024-05-09T20:40:28.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.284+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.92__multipart_key1.2~yA34xoRwHPAIX1AeiluS_V40_u9NCNw.5 2024-05-09T20:40:28.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.292+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.296+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:40:28.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.300+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.300+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.300+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.304+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:40:28.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.304+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.304+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.308+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.308+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:40:28.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.312+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.312+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.312+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:40:28.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.312+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.12877067953724956742', time=2024-05-09T22:36:33.093232+0000, chain.objs.size()=2 2024-05-09T20:40:28.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.312+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.72__shadow_.K5_fBzyTk7N0HYFvl0Y8sP051SpPOlp_1 2024-05-09T20:40:28.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.312+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.72__shadow_.K5_fBzyTk7N0HYFvl0Y8sP051SpPOlp_2 2024-05-09T20:40:28.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.316+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.320+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:40:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.324+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.324+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.324+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:40:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.324+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.18006299820406736526', time=2024-05-09T22:38:12.549273+0000, chain.objs.size()=4 2024-05-09T20:40:28.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.324+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file2.mpg.2~JRyONw0Wdj6A1yOGdYt4byXg-gIaqnD.1 2024-05-09T20:40:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.324+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file2.mpg.2~JRyONw0Wdj6A1yOGdYt4byXg-gIaqnD.2 2024-05-09T20:40:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.324+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file2.mpg.2~JRyONw0Wdj6A1yOGdYt4byXg-gIaqnD.3 2024-05-09T20:40:28.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.324+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.89__multipart_file2.mpg.2~JRyONw0Wdj6A1yOGdYt4byXg-gIaqnD.4 2024-05-09T20:40:28.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.328+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.332+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:40:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:40:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.16477333022078384066', time=2024-05-09T22:37:58.015174+0000, chain.objs.size()=5 2024-05-09T20:40:28.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.87__multipart_key1.2~4C_xf7F_Ufj_ZmmtACqgcnrXK_9YHXC.1 2024-05-09T20:40:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.87__multipart_key1.2~4C_xf7F_Ufj_ZmmtACqgcnrXK_9YHXC.2 2024-05-09T20:40:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.87__multipart_key1.2~4C_xf7F_Ufj_ZmmtACqgcnrXK_9YHXC.3 2024-05-09T20:40:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.87__multipart_key1.2~4C_xf7F_Ufj_ZmmtACqgcnrXK_9YHXC.4 2024-05-09T20:40:28.334 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.336+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.87__multipart_key1.2~4C_xf7F_Ufj_ZmmtACqgcnrXK_9YHXC.5 2024-05-09T20:40:28.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.340+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.348+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:40:28.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.348+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.348+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.352+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.352+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:40:28.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.356+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.356+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.353 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.356+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.360+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:40:28.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.360+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.364+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.364+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.364+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:40:28.364 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17733207191458060046', time=2024-05-09T22:38:04.810054+0000, chain.objs.size()=5 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.88__multipart_key1.2~p2LjciD_ww-Cyny7rtHa-UUYx6ITP0q.1 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.88__multipart_key1.2~p2LjciD_ww-Cyny7rtHa-UUYx6ITP0q.2 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.88__multipart_key1.2~p2LjciD_ww-Cyny7rtHa-UUYx6ITP0q.3 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.88__multipart_key1.2~p2LjciD_ww-Cyny7rtHa-UUYx6ITP0q.4 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.88__multipart_key1.2~p2LjciD_ww-Cyny7rtHa-UUYx6ITP0q.5 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.6681783421231631532', time=2024-05-09T22:38:34.120211+0000, chain.objs.size()=5 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~2Tvm7skCAZ0m_wxjrCzcjCTCE1k8uCI.1 2024-05-09T20:40:28.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__shadow_key1.2~2Tvm7skCAZ0m_wxjrCzcjCTCE1k8uCI.1_1 2024-05-09T20:40:28.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~2Tvm7skCAZ0m_wxjrCzcjCTCE1k8uCI.2 2024-05-09T20:40:28.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__shadow_key1.2~2Tvm7skCAZ0m_wxjrCzcjCTCE1k8uCI.2_1 2024-05-09T20:40:28.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.368+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~2Tvm7skCAZ0m_wxjrCzcjCTCE1k8uCI.3 2024-05-09T20:40:28.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.380+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.384+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:40:28.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:40:28.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1628584102300534071', time=2024-05-09T22:38:56.699356+0000, chain.objs.size()=5 2024-05-09T20:40:28.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.93__multipart_key1.2~_kXyl0u8dgW5UJ3pA50M98ro58VG11f.1 2024-05-09T20:40:28.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.93__multipart_key1.2~_kXyl0u8dgW5UJ3pA50M98ro58VG11f.2 2024-05-09T20:40:28.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.93__multipart_key1.2~_kXyl0u8dgW5UJ3pA50M98ro58VG11f.3 2024-05-09T20:40:28.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.93__multipart_key1.2~_kXyl0u8dgW5UJ3pA50M98ro58VG11f.4 2024-05-09T20:40:28.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.388+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.93__multipart_key1.2~_kXyl0u8dgW5UJ3pA50M98ro58VG11f.5 2024-05-09T20:40:28.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.392+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.400+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:40:28.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.404+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.404+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.404+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.404+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:40:28.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.408+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.408+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2024-05-09T20:40:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.3461174446882225898', time=2024-05-09T22:36:54.616678+0000, chain.objs.size()=5 2024-05-09T20:40:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.1 2024-05-09T20:40:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.2 2024-05-09T20:40:28.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.3 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.4 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.5 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.14984116400385962765', time=2024-05-09T22:37:19.839598+0000, chain.objs.size()=5 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~487Ubuv3fzQMd4gOmQPtBVD0MHbkYof.1 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~487Ubuv3fzQMd4gOmQPtBVD0MHbkYof.2 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~487Ubuv3fzQMd4gOmQPtBVD0MHbkYof.3 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~487Ubuv3fzQMd4gOmQPtBVD0MHbkYof.4 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~487Ubuv3fzQMd4gOmQPtBVD0MHbkYof.5 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.9918309114884531040', time=2024-05-09T22:37:45.098843+0000, chain.objs.size()=5 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~Tbglp2KqhGpp-cyYhuGHvwGrlOOAKox.1 2024-05-09T20:40:28.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.412+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__shadow_key1.2~Tbglp2KqhGpp-cyYhuGHvwGrlOOAKox.1_1 2024-05-09T20:40:28.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.416+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~Tbglp2KqhGpp-cyYhuGHvwGrlOOAKox.2 2024-05-09T20:40:28.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.420+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__shadow_key1.2~Tbglp2KqhGpp-cyYhuGHvwGrlOOAKox.2_1 2024-05-09T20:40:28.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.420+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~Tbglp2KqhGpp-cyYhuGHvwGrlOOAKox.3 2024-05-09T20:40:28.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.420+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10595864983529133311', time=2024-05-09T22:37:51.598853+0000, chain.objs.size()=5 2024-05-09T20:40:28.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.420+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~1wdwqjdhpAFX61fLkuBc88jKF_CS3ul.1 2024-05-09T20:40:28.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.420+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__shadow_key1.2~1wdwqjdhpAFX61fLkuBc88jKF_CS3ul.1_1 2024-05-09T20:40:28.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.420+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~1wdwqjdhpAFX61fLkuBc88jKF_CS3ul.2 2024-05-09T20:40:28.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.420+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__shadow_key1.2~1wdwqjdhpAFX61fLkuBc88jKF_CS3ul.2_1 2024-05-09T20:40:28.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.420+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~1wdwqjdhpAFX61fLkuBc88jKF_CS3ul.3 2024-05-09T20:40:28.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.428+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.436+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:40:28.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.440+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.440+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.440+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.440+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:40:28.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='2~SIU1y_sL3RiUsMfG0iIsGwNOEnXWBNR', time=2024-05-09T22:36:26.416371+0000, chain.objs.size()=5 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.68__multipart_key1.2~SIU1y_sL3RiUsMfG0iIsGwNOEnXWBNR.1 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.68__multipart_key1.2~SIU1y_sL3RiUsMfG0iIsGwNOEnXWBNR.2 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.68__multipart_key1.2~SIU1y_sL3RiUsMfG0iIsGwNOEnXWBNR.3 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.68__multipart_key1.2~SIU1y_sL3RiUsMfG0iIsGwNOEnXWBNR.4 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.68__multipart_key1.2~SIU1y_sL3RiUsMfG0iIsGwNOEnXWBNR.5 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.10463858540847726148', time=2024-05-09T22:37:35.450328+0000, chain.objs.size()=5 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~nkLBaYHqx9Toqen8Ov-yNbq7-GV5cbC.1 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.444+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~nkLBaYHqx9Toqen8Ov-yNbq7-GV5cbC.2 2024-05-09T20:40:28.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.448+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~nkLBaYHqx9Toqen8Ov-yNbq7-GV5cbC.3 2024-05-09T20:40:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.448+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~nkLBaYHqx9Toqen8Ov-yNbq7-GV5cbC.4 2024-05-09T20:40:28.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.448+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~nkLBaYHqx9Toqen8Ov-yNbq7-GV5cbC.5 2024-05-09T20:40:28.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.452+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.460+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:40:28.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.460+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.460+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2024-05-09T20:40:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.1743755973247282477', time=2024-05-09T22:37:50.953340+0000, chain.objs.size()=4 2024-05-09T20:40:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.82__shadow_.NYDybWVxM7N2Xp79GKBw_ZnuTm1WxZe_1 2024-05-09T20:40:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.82__shadow_.NYDybWVxM7N2Xp79GKBw_ZnuTm1WxZe_2 2024-05-09T20:40:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.82__shadow_.NYDybWVxM7N2Xp79GKBw_ZnuTm1WxZe_3 2024-05-09T20:40:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.82__shadow_.NYDybWVxM7N2Xp79GKBw_ZnuTm1WxZe_4 2024-05-09T20:40:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m', time=2024-05-09T22:39:13.895979+0000, chain.objs.size()=1 2024-05-09T20:40:28.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.20 2024-05-09T20:40:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m', time=2024-05-09T22:39:14.188362+0000, chain.objs.size()=59 2024-05-09T20:40:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.1 2024-05-09T20:40:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.2 2024-05-09T20:40:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.3 2024-05-09T20:40:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.4 2024-05-09T20:40:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.5 2024-05-09T20:40:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.6 2024-05-09T20:40:28.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.464+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.7 2024-05-09T20:40:28.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.468+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.8 2024-05-09T20:40:28.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.468+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.9 2024-05-09T20:40:28.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.468+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.10 2024-05-09T20:40:28.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.468+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.11 2024-05-09T20:40:28.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.468+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.12 2024-05-09T20:40:28.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.472+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.13 2024-05-09T20:40:28.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.472+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.14 2024-05-09T20:40:28.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.472+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.15 2024-05-09T20:40:28.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.472+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.16 2024-05-09T20:40:28.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.472+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.17 2024-05-09T20:40:28.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.472+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.18 2024-05-09T20:40:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.476+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.19 2024-05-09T20:40:28.473 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.476+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.PTiO29SX6zz3OLp0jSxkWTryCDnC0VMl.20 2024-05-09T20:40:28.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.21 2024-05-09T20:40:28.477 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.22 2024-05-09T20:40:28.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.23 2024-05-09T20:40:28.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.24 2024-05-09T20:40:28.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.25 2024-05-09T20:40:28.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.26 2024-05-09T20:40:28.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.27 2024-05-09T20:40:28.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.28 2024-05-09T20:40:28.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.29 2024-05-09T20:40:28.478 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.30 2024-05-09T20:40:28.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.480+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.31 2024-05-09T20:40:28.480 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.484+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.32 2024-05-09T20:40:28.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.33 2024-05-09T20:40:28.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.34 2024-05-09T20:40:28.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.35 2024-05-09T20:40:28.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.36 2024-05-09T20:40:28.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.37 2024-05-09T20:40:28.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.38 2024-05-09T20:40:28.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.39 2024-05-09T20:40:28.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.40 2024-05-09T20:40:28.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.41 2024-05-09T20:40:28.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.42 2024-05-09T20:40:28.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.43 2024-05-09T20:40:28.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.488+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.44 2024-05-09T20:40:28.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.492+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.45 2024-05-09T20:40:28.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.492+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.46 2024-05-09T20:40:28.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.492+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.47 2024-05-09T20:40:28.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.496+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.48 2024-05-09T20:40:28.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.496+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.49 2024-05-09T20:40:28.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.496+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.50 2024-05-09T20:40:28.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.496+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.51 2024-05-09T20:40:28.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.496+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.52 2024-05-09T20:40:28.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.496+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.53 2024-05-09T20:40:28.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.496+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.54 2024-05-09T20:40:28.493 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.496+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.55 2024-05-09T20:40:28.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.500+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.56 2024-05-09T20:40:28.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.500+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.57 2024-05-09T20:40:28.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.500+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.58 2024-05-09T20:40:28.496 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.500+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.94__multipart_key1.2~bR5lolWCi41FIRh2UzusvVSHLbptQ0m.59 2024-05-09T20:40:28.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.504+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:40:28.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-09T20:40:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.17033343126761323837', time=2024-05-09T22:38:31.072815+0000, chain.objs.size()=5 2024-05-09T20:40:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~HgGJ6X8RRwm66y_ZV9fxCBqDl5re4zS.1 2024-05-09T20:40:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~HgGJ6X8RRwm66y_ZV9fxCBqDl5re4zS.2 2024-05-09T20:40:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~HgGJ6X8RRwm66y_ZV9fxCBqDl5re4zS.3 2024-05-09T20:40:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~HgGJ6X8RRwm66y_ZV9fxCBqDl5re4zS.4 2024-05-09T20:40:28.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.508+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.91__multipart_key1.2~HgGJ6X8RRwm66y_ZV9fxCBqDl5re4zS.5 2024-05-09T20:40:28.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.512+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.516+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:40:28.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.516+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.516+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2024-05-09T20:40:28.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.2_a3870e0a705cbcc1af9a975dfb68ffaa', time=2024-05-09T22:36:53.942895+0000, chain.objs.size()=5 2024-05-09T20:40:28.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.1 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.2 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.3 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.4 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.80__multipart_key1.2~tMA167myln4PovIkSFQlOz-ARzc3l26.5 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 20 garbage collection: RGWGC::process iterating over entry tag='6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.5176322791345667213', time=2024-05-09T22:37:50.627227+0000, chain.objs.size()=5 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~tLH5ADglgewQ70Bqq8LRSmZceibBrzq.1 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__shadow_key1.2~tLH5ADglgewQ70Bqq8LRSmZceibBrzq.1_1 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~tLH5ADglgewQ70Bqq8LRSmZceibBrzq.2 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__shadow_key1.2~tLH5ADglgewQ70Bqq8LRSmZceibBrzq.2_1 2024-05-09T20:40:28.517 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.520+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0.4602.83__multipart_key1.2~tLH5ADglgewQ70Bqq8LRSmZceibBrzq.3 2024-05-09T20:40:28.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.524+0000 7ff2c35ad980 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-09T20:40:28.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.528+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:40:28.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.528+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.528+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.528+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.532+0000 7ff2c35ad980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:40:28.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.532+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.532+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:28.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.532+0000 7ff2c35ad980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:28.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.536+0000 7ff1777e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:28.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.536+0000 7ff2c35ad980 20 remove_watcher() i=0 2024-05-09T20:40:28.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.536+0000 7ff2c35ad980 2 removed watcher, disabling cache 2024-05-09T20:40:28.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.540+0000 7ff2c35ad980 20 remove_watcher() i=1 2024-05-09T20:40:28.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.540+0000 7ff2c35ad980 20 remove_watcher() i=2 2024-05-09T20:40:28.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.540+0000 7ff2c35ad980 20 remove_watcher() i=3 2024-05-09T20:40:28.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.544+0000 7ff2c35ad980 20 remove_watcher() i=4 2024-05-09T20:40:28.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.544+0000 7ff2c35ad980 20 remove_watcher() i=5 2024-05-09T20:40:28.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.544+0000 7ff2c35ad980 20 remove_watcher() i=6 2024-05-09T20:40:28.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.548+0000 7ff2c35ad980 20 remove_watcher() i=7 2024-05-09T20:40:28.557 DEBUG:teuthology.orchestra.run.smithi063:> radosgw-admin gc process --include-all 2024-05-09T20:40:28.629 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:28.629 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:28.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.676+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 realm 2024-05-09T20:40:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.680+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.684+0000 7f13ed0c1980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.684+0000 7f13ed0c1980 20 searching for the correct realm 2024-05-09T20:40:28.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.688+0000 7f13eabd5640 0 -- 172.21.15.63:0/2127913488 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f13e4017020 msgr2=0x7f13e407c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:28.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.904+0000 7f13eabd5640 0 -- 172.21.15.63:0/2127913488 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f13e4017020 msgr2=0x7f13e407c280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.916+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:28.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 zone default found 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 4 Realm: () 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 zonegroup default 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados->read ofs=0 len=0 2024-05-09T20:40:28.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:28.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 started sync module instance, tier type = 2024-05-09T20:40:28.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.920+0000 7f13ed0c1980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:28.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.944+0000 7f13ed0c1980 20 add_watcher() i=0 2024-05-09T20:40:28.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.944+0000 7f13ed0c1980 20 add_watcher() i=1 2024-05-09T20:40:28.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.944+0000 7f13ed0c1980 20 add_watcher() i=2 2024-05-09T20:40:28.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.944+0000 7f13ed0c1980 20 add_watcher() i=3 2024-05-09T20:40:28.943 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.944+0000 7f13ed0c1980 20 add_watcher() i=4 2024-05-09T20:40:28.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.944+0000 7f13ed0c1980 20 add_watcher() i=5 2024-05-09T20:40:28.944 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.948+0000 7f13ed0c1980 20 add_watcher() i=6 2024-05-09T20:40:28.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.948+0000 7f13ed0c1980 20 add_watcher() i=7 2024-05-09T20:40:28.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.948+0000 7f13ed0c1980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:28.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.956+0000 7f12a17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:28.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.956+0000 7f13ed0c1980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:28.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:28.956+0000 7f13ed0c1980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:29.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.056+0000 7f121bfff640 20 reqs_thread_entry: start 2024-05-09T20:40:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.056+0000 7f13ed0c1980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.056+0000 7f13ed0c1980 20 Filter name: none 2024-05-09T20:40:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.056+0000 7f1219ffb640 20 reqs_thread_entry: start 2024-05-09T20:40:29.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.056+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-09T20:40:29.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.060+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.064+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.064+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.068+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-09T20:40:29.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.068+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.072+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.072+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.076+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-09T20:40:29.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.080+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.080+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.080+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.080 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.084+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-09T20:40:29.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.084+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.088+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.088+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.092+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-09T20:40:29.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.092+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.092+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.096+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.096+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-09T20:40:29.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.100+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.100+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.104+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.102 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.104+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-09T20:40:29.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.108+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.112+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.112+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.116+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-09T20:40:29.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.116+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.116+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.120+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.120+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-09T20:40:29.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.124+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.124+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.128+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.128+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-09T20:40:29.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.132+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.130 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.132+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.132+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.136+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-09T20:40:29.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.140+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.138 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.140+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.140+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.144+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-09T20:40:29.144 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.148+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.148+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.148+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.152+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-09T20:40:29.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.156+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.156+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.156+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.160+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-09T20:40:29.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.164+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.164+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.164+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.168+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-09T20:40:29.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.172+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.172+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.172+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.176+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-09T20:40:29.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.180+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.180+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.180+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.184+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-09T20:40:29.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.184+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.188+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.188+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.188+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-09T20:40:29.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.192+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.192+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.196+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.196+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-09T20:40:29.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.200+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.200+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.200+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.204+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-09T20:40:29.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.208+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.208+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.212+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.216+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-09T20:40:29.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.216+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.220+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.220+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.220+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-09T20:40:29.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.224+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.228+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.228+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.232+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-09T20:40:29.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.232+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.236+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.236+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.236+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-09T20:40:29.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.240+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.244+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.244+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.248+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-09T20:40:29.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.248+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.248+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.252+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.252+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-09T20:40:29.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.256+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.256+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.256+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.260+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-09T20:40:29.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.264+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.264+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.268+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.268+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-09T20:40:29.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.272+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.276+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.276+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.280+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-09T20:40:29.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.280+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.284+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.284+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.288+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-09T20:40:29.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.292+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.292+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.296+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.296+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-09T20:40:29.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.300+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.304+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.304+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.304+0000 7f13ed0c1980 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-09T20:40:29.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.308+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.312+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-09T20:40:29.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.312+0000 7f13ed0c1980 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-09T20:40:29.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.316+0000 7f12a17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:29.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.320+0000 7f13ed0c1980 20 remove_watcher() i=0 2024-05-09T20:40:29.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.320+0000 7f13ed0c1980 2 removed watcher, disabling cache 2024-05-09T20:40:29.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.324+0000 7f13ed0c1980 20 remove_watcher() i=1 2024-05-09T20:40:29.324 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.328+0000 7f13ed0c1980 20 remove_watcher() i=2 2024-05-09T20:40:29.326 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.328+0000 7f13ed0c1980 20 remove_watcher() i=3 2024-05-09T20:40:29.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.332+0000 7f13ed0c1980 20 remove_watcher() i=4 2024-05-09T20:40:29.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.332+0000 7f13ed0c1980 20 remove_watcher() i=5 2024-05-09T20:40:29.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.336+0000 7f13ed0c1980 20 remove_watcher() i=6 2024-05-09T20:40:29.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:29.340+0000 7f13ed0c1980 20 remove_watcher() i=7 2024-05-09T20:40:29.355 INFO:teuthology.run_tasks:Running task s3tests... 2024-05-09T20:40:29.371 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'accounts': {'iam alt root': 'RGW99999999999999999', 'iam root': 'RGW88888888888888888', 's3 main': 'RGW11111111111111111'}, 'force-branch': 'ceph-master', 'slow_backend': True, 'storage classes': 'LUKEWARM, FROZEN'}} 2024-05-09T20:40:29.371 INFO:tasks.s3tests:Downloading s3-tests... 2024-05-09T20:40:29.371 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2024-05-09T20:40:29.371 DEBUG:teuthology.orchestra.run.smithi063:> git clone -b ceph-master https://git.ceph.com/s3-tests.git /home/ubuntu/cephtest/s3-tests-client.0 2024-05-09T20:40:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/s3-tests-client.0'... 2024-05-09T20:40:30.235 INFO:tasks.s3tests:Creating rgw users... 2024-05-09T20:40:30.235 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW11111111111111111 --account-name Mr.RGW11111111111111111 --email RGW11111111111111111+test@test.test 2024-05-09T20:40:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:30.321 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:30.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 20 realm 2024-05-09T20:40:30.365 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:30.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.368+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.372+0000 7f9dad6f7980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:30.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.372+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.372+0000 7f9dad6f7980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:30.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.372+0000 7f9dad6f7980 20 searching for the correct realm 2024-05-09T20:40:30.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.372+0000 7f9daaa0a640 0 -- 172.21.15.63:0/3838318770 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f9d9c101c20 msgr2=0x7f9d9c0a5040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:30.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.584+0000 7f9daaa0a640 0 -- 172.21.15.63:0/3838318770 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f9d9c0f70c0 msgr2=0x7f9d9c0dd1d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:30.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:30.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:30.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:30.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:30.805 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:30.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:30.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:30.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.808+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:30.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 zone default found 2024-05-09T20:40:30.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 4 Realm: () 2024-05-09T20:40:30.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:30.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:30.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:30.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 zonegroup default 2024-05-09T20:40:30.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 started sync module instance, tier type = 2024-05-09T20:40:30.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.812+0000 7f9dad6f7980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:30.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 20 add_watcher() i=0 2024-05-09T20:40:30.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 20 add_watcher() i=1 2024-05-09T20:40:30.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 20 add_watcher() i=2 2024-05-09T20:40:30.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 20 add_watcher() i=3 2024-05-09T20:40:30.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 20 add_watcher() i=4 2024-05-09T20:40:30.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 20 add_watcher() i=5 2024-05-09T20:40:30.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 20 add_watcher() i=6 2024-05-09T20:40:30.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 20 add_watcher() i=7 2024-05-09T20:40:30.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.836+0000 7f9dad6f7980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:30.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.840+0000 7f9c61fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:30.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.840+0000 7f9dad6f7980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:30.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.840+0000 7f9dad6f7980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:30.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.840+0000 7f9dad6f7980 5 note: GC not initialized 2024-05-09T20:40:30.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.840+0000 7f9c60fe1640 20 reqs_thread_entry: start 2024-05-09T20:40:30.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.840+0000 7f9dad6f7980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:30.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 20 Filter name: none 2024-05-09T20:40:30.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9bda7fc640 20 reqs_thread_entry: start 2024-05-09T20:40:30.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 : miss 2024-05-09T20:40:30.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x0 2024-05-09T20:40:30.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 adding default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-05-09T20:40:30.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 20 failed to read name.$Mr.RGW11111111111111111 with: (2) No such file or directory 2024-05-09T20:40:30.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 cache get: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test : miss 2024-05-09T20:40:30.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 20 rados->read ofs=0 len=0 2024-05-09T20:40:30.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:30.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x0 2024-05-09T20:40:30.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 adding default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-05-09T20:40:30.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 20 failed to read rgw11111111111111111+test@test.test with: (2) No such file or directory 2024-05-09T20:40:30.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-05-09T20:40:30.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 adding default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-05-09T20:40:30.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.844+0000 7f9dad6f7980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW11111111111111111, ofs0, ns] 2024-05-09T20:40:30.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.848+0000 7f9c8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431681 cookie 93978911502272 notifier 7154 bl.length()=384 2024-05-09T20:40:30.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.848+0000 7f9c8effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-05-09T20:40:30.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.848+0000 7f9c8effd640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-05-09T20:40:30.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.852+0000 7f9dad6f7980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x17 2024-05-09T20:40:30.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.852+0000 7f9dad6f7980 10 moving default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-05-09T20:40:30.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.852+0000 7f9dad6f7980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW11111111111111111, ofs0, ns] 2024-05-09T20:40:30.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.852+0000 7f9c8effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431686 cookie 93978911514480 notifier 7154 bl.length()=225 2024-05-09T20:40:30.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.852+0000 7f9c8effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 info.flags=0x17 2024-05-09T20:40:30.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.852+0000 7f9c8effd640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW11111111111111111 to cache LRU end 2024-05-09T20:40:30.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.856+0000 7f9dad6f7980 10 cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x17 2024-05-09T20:40:30.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.856+0000 7f9dad6f7980 10 moving default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-05-09T20:40:30.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.856+0000 7f9dad6f7980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw11111111111111111+test@test.test, ofs0, ns] 2024-05-09T20:40:30.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.856+0000 7f9c8e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431645 cookie 93978911512352 notifier 7154 bl.length()=234 2024-05-09T20:40:30.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.856+0000 7f9c8e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw11111111111111111+test@test.test info.flags=0x17 2024-05-09T20:40:30.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.856+0000 7f9c8e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw11111111111111111+test@test.test to cache LRU end 2024-05-09T20:40:30.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.860+0000 7f9c61fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:30.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.868+0000 7f9dad6f7980 20 remove_watcher() i=0 2024-05-09T20:40:30.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.868+0000 7f9dad6f7980 2 removed watcher, disabling cache 2024-05-09T20:40:30.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.868+0000 7f9dad6f7980 20 remove_watcher() i=1 2024-05-09T20:40:30.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.872+0000 7f9dad6f7980 20 remove_watcher() i=2 2024-05-09T20:40:30.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.872+0000 7f9dad6f7980 20 remove_watcher() i=3 2024-05-09T20:40:30.872 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.876+0000 7f9dad6f7980 20 remove_watcher() i=4 2024-05-09T20:40:30.874 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.876+0000 7f9dad6f7980 20 remove_watcher() i=5 2024-05-09T20:40:30.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.880+0000 7f9dad6f7980 20 remove_watcher() i=6 2024-05-09T20:40:30.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:30.880+0000 7f9dad6f7980 20 remove_watcher() i=7 2024-05-09T20:40:30.891 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:30.891 INFO:teuthology.orchestra.run.smithi063.stdout: "id": "RGW11111111111111111", 2024-05-09T20:40:30.891 INFO:teuthology.orchestra.run.smithi063.stdout: "tenant": "", 2024-05-09T20:40:30.891 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "Mr.RGW11111111111111111", 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "RGW11111111111111111+test@test.test", 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "quota": { 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:30.892 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:30.893 INFO:teuthology.orchestra.run.smithi063.stdout: "max_users": 1000, 2024-05-09T20:40:30.893 INFO:teuthology.orchestra.run.smithi063.stdout: "max_roles": 1000, 2024-05-09T20:40:30.893 INFO:teuthology.orchestra.run.smithi063.stdout: "max_groups": 1000, 2024-05-09T20:40:30.893 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:30.893 INFO:teuthology.orchestra.run.smithi063.stdout: "max_access_keys": 4 2024-05-09T20:40:30.893 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:30.895 DEBUG:tasks.s3tests:Creating user RGW11111111111111111 on client.0 2024-05-09T20:40:30.895 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name Mr.RGW11111111111111111 --email foo+test@test.test --access-key NQYPJDIRJITVGIOTFYST --secret Y0b8pPnji8pWHB2jeVfKUfiCFulj/csQRxA0Zy+nRR1ufi0YbBJhYQ== --cluster ceph --account-id RGW11111111111111111 --account-root 2024-05-09T20:40:30.984 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:30.985 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:31.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.028+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 realm 2024-05-09T20:40:31.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:31.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.028 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:31.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:31.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.032+0000 7fc42ca4b980 20 searching for the correct realm 2024-05-09T20:40:31.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.036+0000 7fc42a55f640 0 -- 172.21.15.63:0/1221530957 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fc42401f070 msgr2=0x7fc424021a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:31.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.252+0000 7fc42a55f640 0 -- 172.21.15.63:0/1221530957 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fc42401f070 msgr2=0x7fc424021a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.264+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:31.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 zone default found 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 4 Realm: () 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 zonegroup default 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 started sync module instance, tier type = 2024-05-09T20:40:31.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.268+0000 7fc42ca4b980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:31.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 20 add_watcher() i=0 2024-05-09T20:40:31.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 20 add_watcher() i=1 2024-05-09T20:40:31.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 20 add_watcher() i=2 2024-05-09T20:40:31.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 20 add_watcher() i=3 2024-05-09T20:40:31.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 20 add_watcher() i=4 2024-05-09T20:40:31.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 20 add_watcher() i=5 2024-05-09T20:40:31.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 20 add_watcher() i=6 2024-05-09T20:40:31.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 20 add_watcher() i=7 2024-05-09T20:40:31.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.296+0000 7fc42ca4b980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.300+0000 7fc2e17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.300+0000 7fc42ca4b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.300+0000 7fc42ca4b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.300+0000 7fc42ca4b980 5 note: GC not initialized 2024-05-09T20:40:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc2db7fe640 20 reqs_thread_entry: start 2024-05-09T20:40:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 Filter name: none 2024-05-09T20:40:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc2d97fa640 20 reqs_thread_entry: start 2024-05-09T20:40:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-05-09T20:40:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2024-05-09T20:40:31.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-09T20:40:31.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache get: name=default.rgw.meta+users.email+foo+test@test.test : miss 2024-05-09T20:40:31.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x0 2024-05-09T20:40:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 adding default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-05-09T20:40:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache get: name=default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST : miss 2024-05-09T20:40:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache put: name=default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST info.flags=0x0 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 adding default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST to cache LRU end 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache get: name=default.rgw.meta+accounts+account.RGW11111111111111111 : miss 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 20 rados_obj.operate() r=0 bl.length=184 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache put: name=default.rgw.meta+accounts+account.RGW11111111111111111 info.flags=0x17 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 adding default.rgw.meta+accounts+account.RGW11111111111111111 to cache LRU end 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache get: name=default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST : hit (negative entry) 2024-05-09T20:40:31.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.304+0000 7fc42ca4b980 10 cache get: name=default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST : hit (negative entry) 2024-05-09T20:40:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.308+0000 7fc42ca4b980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-05-09T20:40:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.308+0000 7fc42ca4b980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-09T20:40:31.305 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.308+0000 7fc42ca4b980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-05-09T20:40:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.308+0000 7fc30dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431687 cookie 94022969445104 notifier 7638 bl.length()=673 2024-05-09T20:40:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.308+0000 7fc30dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-05-09T20:40:31.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.308+0000 7fc30dffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-09T20:40:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.312+0000 7fc42ca4b980 10 cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-05-09T20:40:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.312+0000 7fc42ca4b980 10 moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-05-09T20:40:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.312+0000 7fc42ca4b980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.email:foo+test@test.test, ofs0, ns] 2024-05-09T20:40:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.316+0000 7fc30dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431688 cookie 94022969457920 notifier 7638 bl.length()=185 2024-05-09T20:40:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.316+0000 7fc30dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo+test@test.test info.flags=0x7 2024-05-09T20:40:31.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.316+0000 7fc30dffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo+test@test.test to cache LRU end 2024-05-09T20:40:31.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.320+0000 7fc42ca4b980 10 cache put: name=default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST info.flags=0x7 2024-05-09T20:40:31.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.320+0000 7fc42ca4b980 10 moving default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST to cache LRU end 2024-05-09T20:40:31.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.320+0000 7fc42ca4b980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NQYPJDIRJITVGIOTFYST, ofs0, ns] 2024-05-09T20:40:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.320+0000 7fc30dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1125281431730 cookie 94022969459248 notifier 7638 bl.length()=186 2024-05-09T20:40:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.320+0000 7fc30dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST info.flags=0x7 2024-05-09T20:40:31.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.320+0000 7fc30dffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NQYPJDIRJITVGIOTFYST to cache LRU end 2024-05-09T20:40:31.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.324+0000 7fc42ca4b980 20 linked user foo.client.0 to account RGW11111111111111111 2024-05-09T20:40:31.322 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:31.322 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "foo.client.0", 2024-05-09T20:40:31.322 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr.RGW11111111111111111", 2024-05-09T20:40:31.322 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "foo+test@test.test", 2024-05-09T20:40:31.322 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "foo.client.0", 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "NQYPJDIRJITVGIOTFYST", 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "Y0b8pPnji8pWHB2jeVfKUfiCFulj/csQRxA0Zy+nRR1ufi0YbBJhYQ==", 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:31.309646Z" 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:40:31.323 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:31.324 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "root", 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "RGW11111111111111111", 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:31.309607Z", 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:31.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.328+0000 7fc2e17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.332+0000 7fc42ca4b980 20 remove_watcher() i=0 2024-05-09T20:40:31.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.332+0000 7fc42ca4b980 2 removed watcher, disabling cache 2024-05-09T20:40:31.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.332+0000 7fc42ca4b980 20 remove_watcher() i=1 2024-05-09T20:40:31.333 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.336+0000 7fc42ca4b980 20 remove_watcher() i=2 2024-05-09T20:40:31.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.340+0000 7fc42ca4b980 20 remove_watcher() i=3 2024-05-09T20:40:31.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.340+0000 7fc42ca4b980 20 remove_watcher() i=4 2024-05-09T20:40:31.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.344+0000 7fc42ca4b980 20 remove_watcher() i=5 2024-05-09T20:40:31.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.344+0000 7fc42ca4b980 20 remove_watcher() i=6 2024-05-09T20:40:31.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.344+0000 7fc42ca4b980 20 remove_watcher() i=7 2024-05-09T20:40:31.360 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 4990206863 --totp-seed 56OEWLTIJZNIW3OEKSTZSONUDNHLTWFRBJOJZI5VSYZZYCKXOG2XQGBIT4POI2ZW --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-09T20:40:31.401 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:31.401 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:31.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.444+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.444+0000 7f05b0a4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.444+0000 7f05b0a4f980 20 realm 2024-05-09T20:40:31.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.444+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:31.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.445 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:31.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:31.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.448+0000 7f05b0a4f980 20 searching for the correct realm 2024-05-09T20:40:31.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.452+0000 7f05aed64640 0 -- 172.21.15.63:0/3449528133 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f05a40d2070 msgr2=0x7f05a40c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:31.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.664+0000 7f05add62640 0 -- 172.21.15.63:0/3449528133 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f05a007f040 msgr2=0x7f05a00542e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:31.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:31.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:31.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:31.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:31.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:31.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:31.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.681 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:31.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 zone default found 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 4 Realm: () 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 zonegroup default 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.684+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.688+0000 7f05b0a4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.688+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.688+0000 7f05b0a4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.688+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.688+0000 7f05b0a4f980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.688+0000 7f05b0a4f980 20 started sync module instance, tier type = 2024-05-09T20:40:31.685 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.688+0000 7f05b0a4f980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.712+0000 7f05b0a4f980 20 add_watcher() i=0 2024-05-09T20:40:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.712+0000 7f05b0a4f980 20 add_watcher() i=1 2024-05-09T20:40:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.712+0000 7f05b0a4f980 20 add_watcher() i=2 2024-05-09T20:40:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.712+0000 7f05b0a4f980 20 add_watcher() i=3 2024-05-09T20:40:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.712+0000 7f05b0a4f980 20 add_watcher() i=4 2024-05-09T20:40:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.712+0000 7f05b0a4f980 20 add_watcher() i=5 2024-05-09T20:40:31.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.712+0000 7f05b0a4f980 20 add_watcher() i=6 2024-05-09T20:40:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.712+0000 7f05b0a4f980 20 add_watcher() i=7 2024-05-09T20:40:31.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.716+0000 7f05b0a4f980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.716+0000 7f0464fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:31.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 5 note: GC not initialized 2024-05-09T20:40:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f03df7fe640 20 reqs_thread_entry: start 2024-05-09T20:40:31.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 20 Filter name: none 2024-05-09T20:40:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f03dd7fa640 20 reqs_thread_entry: start 2024-05-09T20:40:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2024-05-09T20:40:31.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 20 rados->read ofs=0 len=0 2024-05-09T20:40:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 20 rados_obj.operate() r=0 bl.length=488 2024-05-09T20:40:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x13 2024-05-09T20:40:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-09T20:40:31.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:31.720+0000 7f05b0a4f980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:33.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.532+0000 7f05b0a4f980 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-05-09T20:40:33.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.532+0000 7f05b0a4f980 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-09T20:40:33.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.532+0000 7f05b0a4f980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:foo.client.0, ofs0, ns] 2024-05-09T20:40:33.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.536+0000 7f04917fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366281 cookie 93957724146944 notifier 7190 bl.length()=687 2024-05-09T20:40:33.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.536+0000 7f04917fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2024-05-09T20:40:33.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.536+0000 7f04917fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2024-05-09T20:40:33.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.540+0000 7f0464fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:33.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.548+0000 7f05b0a4f980 20 remove_watcher() i=0 2024-05-09T20:40:33.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.548+0000 7f05b0a4f980 2 removed watcher, disabling cache 2024-05-09T20:40:33.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.548+0000 7f05b0a4f980 20 remove_watcher() i=1 2024-05-09T20:40:33.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.552+0000 7f05b0a4f980 20 remove_watcher() i=2 2024-05-09T20:40:33.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.552+0000 7f05b0a4f980 20 remove_watcher() i=3 2024-05-09T20:40:33.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.556+0000 7f05b0a4f980 20 remove_watcher() i=4 2024-05-09T20:40:33.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.556+0000 7f05b0a4f980 20 remove_watcher() i=5 2024-05-09T20:40:33.556 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.560+0000 7f05b0a4f980 20 remove_watcher() i=6 2024-05-09T20:40:33.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.560+0000 7f05b0a4f980 20 remove_watcher() i=7 2024-05-09T20:40:33.583 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2024-05-09T20:40:33.583 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name Mr.bar.client.0 --email bar+test@test.test --access-key DQTKAOSUYHBITDERRLVH --secret YV3GxRlDQfbc6Pf7LJfp03sCl0v0XCtGT/2ifgEU66zIpXj3fu95wQ== --cluster ceph 2024-05-09T20:40:33.674 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:33.674 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.720+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.720+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.720+0000 7fc678a72980 20 realm 2024-05-09T20:40:33.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.720+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:33.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:33.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:33.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.724+0000 7fc678a72980 20 searching for the correct realm 2024-05-09T20:40:33.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.728+0000 7fc676d87640 0 -- 172.21.15.63:0/540857187 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fc66007dc20 msgr2=0x7fc660074ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:33.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.736+0000 7fc676d87640 0 -- 172.21.15.63:0/540857187 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fc5400781f0 msgr2=0x7fc54007a620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:40:33.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:33.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:33.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 zone default found 2024-05-09T20:40:33.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 4 Realm: () 2024-05-09T20:40:33.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:33.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:33.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:33.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 zonegroup default 2024-05-09T20:40:33.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.748+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.752+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.752+0000 7fc678a72980 20 started sync module instance, tier type = 2024-05-09T20:40:33.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.752+0000 7fc678a72980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:33.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.764+0000 7fc678a72980 20 add_watcher() i=0 2024-05-09T20:40:33.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.764+0000 7fc678a72980 20 add_watcher() i=1 2024-05-09T20:40:33.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.764+0000 7fc678a72980 20 add_watcher() i=2 2024-05-09T20:40:33.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.764+0000 7fc678a72980 20 add_watcher() i=3 2024-05-09T20:40:33.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.764+0000 7fc678a72980 20 add_watcher() i=4 2024-05-09T20:40:33.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.764+0000 7fc678a72980 20 add_watcher() i=5 2024-05-09T20:40:33.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.764+0000 7fc678a72980 20 add_watcher() i=6 2024-05-09T20:40:33.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.768+0000 7fc678a72980 20 add_watcher() i=7 2024-05-09T20:40:33.764 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.768+0000 7fc678a72980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:33.767 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.768+0000 7fc52cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.768+0000 7fc678a72980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.768+0000 7fc678a72980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.768+0000 7fc678a72980 5 note: GC not initialized 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.768+0000 7fc4a77fe640 20 reqs_thread_entry: start 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 20 Filter name: none 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc4a57fa640 20 reqs_thread_entry: start 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-05-09T20:40:33.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2024-05-09T20:40:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-09T20:40:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 cache get: name=default.rgw.meta+users.email+bar+test@test.test : miss 2024-05-09T20:40:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x0 2024-05-09T20:40:33.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 adding default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-05-09T20:40:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 cache get: name=default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH : miss 2024-05-09T20:40:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 cache put: name=default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH info.flags=0x0 2024-05-09T20:40:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 adding default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH to cache LRU end 2024-05-09T20:40:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 cache get: name=default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH : hit (negative entry) 2024-05-09T20:40:33.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.772+0000 7fc678a72980 10 cache get: name=default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH : hit (negative entry) 2024-05-09T20:40:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.776+0000 7fc678a72980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-05-09T20:40:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.776+0000 7fc678a72980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-09T20:40:33.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.776+0000 7fc678a72980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-05-09T20:40:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.776+0000 7fc559ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366274 cookie 94350610187888 notifier 7208 bl.length()=645 2024-05-09T20:40:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.776+0000 7fc559ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-05-09T20:40:33.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.776+0000 7fc559ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-09T20:40:33.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.780+0000 7fc678a72980 10 cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-05-09T20:40:33.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.780+0000 7fc678a72980 10 moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-05-09T20:40:33.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.780+0000 7fc678a72980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:bar+test@test.test, ofs0, ns] 2024-05-09T20:40:33.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.780+0000 7fc559ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366323 cookie 94350610197712 notifier 7208 bl.length()=185 2024-05-09T20:40:33.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.780+0000 7fc559ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+bar+test@test.test info.flags=0x7 2024-05-09T20:40:33.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.780+0000 7fc559ffb640 10 rgw watcher librados: moving default.rgw.meta+users.email+bar+test@test.test to cache LRU end 2024-05-09T20:40:33.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.784+0000 7fc678a72980 10 cache put: name=default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH info.flags=0x7 2024-05-09T20:40:33.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.784+0000 7fc678a72980 10 moving default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH to cache LRU end 2024-05-09T20:40:33.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.784+0000 7fc678a72980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:DQTKAOSUYHBITDERRLVH, ofs0, ns] 2024-05-09T20:40:33.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.784+0000 7fc559ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366282 cookie 94350610193936 notifier 7208 bl.length()=186 2024-05-09T20:40:33.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.784+0000 7fc559ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH info.flags=0x7 2024-05-09T20:40:33.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.784+0000 7fc559ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+DQTKAOSUYHBITDERRLVH to cache LRU end 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "bar.client.0", 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr.bar.client.0", 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "bar+test@test.test", 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "bar.client.0", 2024-05-09T20:40:33.783 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "DQTKAOSUYHBITDERRLVH", 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "YV3GxRlDQfbc6Pf7LJfp03sCl0v0XCtGT/2ifgEU66zIpXj3fu95wQ==", 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:33.775815Z" 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:33.784 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "rgw", 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "", 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:33.775795Z", 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:40:33.785 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:40:33.786 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:33.786 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:33.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.788+0000 7fc52cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:33.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.792+0000 7fc678a72980 20 remove_watcher() i=0 2024-05-09T20:40:33.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.792+0000 7fc678a72980 2 removed watcher, disabling cache 2024-05-09T20:40:33.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.792+0000 7fc678a72980 20 remove_watcher() i=1 2024-05-09T20:40:33.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.796+0000 7fc678a72980 20 remove_watcher() i=2 2024-05-09T20:40:33.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.796+0000 7fc678a72980 20 remove_watcher() i=3 2024-05-09T20:40:33.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.800+0000 7fc678a72980 20 remove_watcher() i=4 2024-05-09T20:40:33.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.800+0000 7fc678a72980 20 remove_watcher() i=5 2024-05-09T20:40:33.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.804+0000 7fc678a72980 20 remove_watcher() i=6 2024-05-09T20:40:33.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.804+0000 7fc678a72980 20 remove_watcher() i=7 2024-05-09T20:40:33.816 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 3928061448 --totp-seed VYPNAQ26RGV3HO6CUGLTVWVUBJPS75TN4RF7GF56I6SIFJYXTG4YGZ24RQKZXXMR --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-09T20:40:33.861 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:33.861 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:33.900 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 20 realm 2024-05-09T20:40:33.902 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:33.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:33.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.904+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.908+0000 7f2e7b718980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:33.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.908+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:33.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.908+0000 7f2e7b718980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:33.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.908+0000 7f2e7b718980 20 searching for the correct realm 2024-05-09T20:40:33.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:33.912+0000 7f2e7922c640 0 -- 172.21.15.63:0/1245468228 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f2e740a9070 msgr2=0x7f2e740efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:40:34.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.124+0000 7f2e7922c640 0 -- 172.21.15.63:0/1245468228 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f2e740a9070 msgr2=0x7f2e740efac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:34.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:34.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:34.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:34.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:34.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:34.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:34.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:34.134 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:34.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 zone default found 2024-05-09T20:40:34.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 4 Realm: () 2024-05-09T20:40:34.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:34.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:34.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:34.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 zonegroup default 2024-05-09T20:40:34.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.136+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.140+0000 7f2e7b718980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.140+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.140+0000 7f2e7b718980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.140+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.140+0000 7f2e7b718980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.140+0000 7f2e7b718980 20 started sync module instance, tier type = 2024-05-09T20:40:34.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.140+0000 7f2e7b718980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:34.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 20 add_watcher() i=0 2024-05-09T20:40:34.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 20 add_watcher() i=1 2024-05-09T20:40:34.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 20 add_watcher() i=2 2024-05-09T20:40:34.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 20 add_watcher() i=3 2024-05-09T20:40:34.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 20 add_watcher() i=4 2024-05-09T20:40:34.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 20 add_watcher() i=5 2024-05-09T20:40:34.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 20 add_watcher() i=6 2024-05-09T20:40:34.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 20 add_watcher() i=7 2024-05-09T20:40:34.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.160+0000 7f2e7b718980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2d2f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2e7b718980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2e7b718980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2e7b718980 5 note: GC not initialized 2024-05-09T20:40:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2d2e7e4640 20 reqs_thread_entry: start 2024-05-09T20:40:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2e7b718980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2e7b718980 20 Filter name: none 2024-05-09T20:40:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2c9ffff640 20 reqs_thread_entry: start 2024-05-09T20:40:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2e7b718980 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2024-05-09T20:40:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.164+0000 7f2e7b718980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.168+0000 7f2e7b718980 20 rados_obj.operate() r=0 bl.length=460 2024-05-09T20:40:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.168+0000 7f2e7b718980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x13 2024-05-09T20:40:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.168+0000 7f2e7b718980 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-09T20:40:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.168+0000 7f2e7b718980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:34.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.172+0000 7f2e7b718980 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-05-09T20:40:34.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.172+0000 7f2e7b718980 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-09T20:40:34.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.172+0000 7f2e7b718980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:bar.client.0, ofs0, ns] 2024-05-09T20:40:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.172+0000 7f2d5cff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1133871366275 cookie 94233052893360 notifier 7226 bl.length()=659 2024-05-09T20:40:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.172+0000 7f2d5cff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2024-05-09T20:40:34.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.172+0000 7f2d5cff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2024-05-09T20:40:34.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.176+0000 7f2d2f7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:34.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.180+0000 7f2e7b718980 20 remove_watcher() i=0 2024-05-09T20:40:34.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.180+0000 7f2e7b718980 2 removed watcher, disabling cache 2024-05-09T20:40:34.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.184+0000 7f2e7b718980 20 remove_watcher() i=1 2024-05-09T20:40:34.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.188+0000 7f2e7b718980 20 remove_watcher() i=2 2024-05-09T20:40:34.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.188+0000 7f2e7b718980 20 remove_watcher() i=3 2024-05-09T20:40:34.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.192+0000 7f2e7b718980 20 remove_watcher() i=4 2024-05-09T20:40:34.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.192+0000 7f2e7b718980 20 remove_watcher() i=5 2024-05-09T20:40:34.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.192+0000 7f2e7b718980 20 remove_watcher() i=6 2024-05-09T20:40:34.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.196+0000 7f2e7b718980 20 remove_watcher() i=7 2024-05-09T20:40:34.210 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2024-05-09T20:40:34.210 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr.testx$tenanteduser.client.0' --email 'testx$tenanteduser+test@test.test' --access-key ZHQWLAITRERLIQILGIUA --secret MT774KXxL56sdlYh32S5O1DDigNmWxGhBZVZLMPtj4+DaRERsFqA6g== --cluster ceph 2024-05-09T20:40:34.250 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:34.250 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:34.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.296+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 realm 2024-05-09T20:40:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:34.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:34.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.300+0000 7f0eae55d980 20 searching for the correct realm 2024-05-09T20:40:34.303 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.308+0000 7f0eac071640 0 -- 172.21.15.63:0/1943945750 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f0ea40d9040 msgr2=0x7f0ea40c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:34.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.564+0000 7f0eac071640 0 -- 172.21.15.63:0/1943945750 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f0ea40d9040 msgr2=0x7f0ea40c3920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:34.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:34.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 zone default found 2024-05-09T20:40:34.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 4 Realm: () 2024-05-09T20:40:34.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:34.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:34.575 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:34.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 zonegroup default 2024-05-09T20:40:34.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.576+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.580+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.580+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.580+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.580+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.580+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.580+0000 7f0eae55d980 20 started sync module instance, tier type = 2024-05-09T20:40:34.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.580+0000 7f0eae55d980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.600+0000 7f0eae55d980 20 add_watcher() i=0 2024-05-09T20:40:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.600+0000 7f0eae55d980 20 add_watcher() i=1 2024-05-09T20:40:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.600+0000 7f0eae55d980 20 add_watcher() i=2 2024-05-09T20:40:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.600+0000 7f0eae55d980 20 add_watcher() i=3 2024-05-09T20:40:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.600+0000 7f0eae55d980 20 add_watcher() i=4 2024-05-09T20:40:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.600+0000 7f0eae55d980 20 add_watcher() i=5 2024-05-09T20:40:34.599 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.600+0000 7f0eae55d980 20 add_watcher() i=6 2024-05-09T20:40:34.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.604+0000 7f0eae55d980 20 add_watcher() i=7 2024-05-09T20:40:34.600 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.604+0000 7f0eae55d980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:34.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0d62fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:34.605 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:34.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:34.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 5 note: GC not initialized 2024-05-09T20:40:34.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0d61fdb640 20 reqs_thread_entry: start 2024-05-09T20:40:34.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:34.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 Filter name: none 2024-05-09T20:40:34.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0cd37fe640 20 reqs_thread_entry: start 2024-05-09T20:40:34.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-05-09T20:40:34.606 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test : miss 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x0 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 10 adding default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 10 cache get: name=default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA : miss 2024-05-09T20:40:34.607 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.608+0000 7f0eae55d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0eae55d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0eae55d980 10 cache put: name=default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA info.flags=0x0 2024-05-09T20:40:34.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0eae55d980 10 adding default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA to cache LRU end 2024-05-09T20:40:34.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0eae55d980 10 cache get: name=default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA : hit (negative entry) 2024-05-09T20:40:34.608 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0eae55d980 10 cache get: name=default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA : hit (negative entry) 2024-05-09T20:40:34.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0eae55d980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-05-09T20:40:34.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0eae55d980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-09T20:40:34.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0eae55d980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-05-09T20:40:34.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0d837fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333620 cookie 94661309854992 notifier 7244 bl.length()=719 2024-05-09T20:40:34.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0d837fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-05-09T20:40:34.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.612+0000 7f0d837fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-09T20:40:34.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.620+0000 7f0eae55d980 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-05-09T20:40:34.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.620+0000 7f0eae55d980 10 moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-05-09T20:40:34.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.620+0000 7f0eae55d980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.email:testx$tenanteduser+test@test.test, ofs0, ns] 2024-05-09T20:40:34.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.620+0000 7f0d837fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333520 cookie 94661309857328 notifier 7244 bl.length()=215 2024-05-09T20:40:34.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.620+0000 7f0d837fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+testx$tenanteduser+test@test.test info.flags=0x7 2024-05-09T20:40:34.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.620+0000 7f0d837fe640 10 rgw watcher librados: moving default.rgw.meta+users.email+testx$tenanteduser+test@test.test to cache LRU end 2024-05-09T20:40:34.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.624+0000 7f0eae55d980 10 cache put: name=default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA info.flags=0x7 2024-05-09T20:40:34.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.624+0000 7f0eae55d980 10 moving default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA to cache LRU end 2024-05-09T20:40:34.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.624+0000 7f0eae55d980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:ZHQWLAITRERLIQILGIUA, ofs0, ns] 2024-05-09T20:40:34.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.624+0000 7f0d82ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333534 cookie 94661309845984 notifier 7244 bl.length()=201 2024-05-09T20:40:34.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.624+0000 7f0d82ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA info.flags=0x7 2024-05-09T20:40:34.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.624+0000 7f0d82ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+ZHQWLAITRERLIQILGIUA to cache LRU end 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "testx$tenanteduser.client.0", 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr.testx$tenanteduser.client.0", 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "testx$tenanteduser+test@test.test", 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "testx$tenanteduser.client.0", 2024-05-09T20:40:34.624 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "ZHQWLAITRERLIQILGIUA", 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "MT774KXxL56sdlYh32S5O1DDigNmWxGhBZVZLMPtj4+DaRERsFqA6g==", 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:34.614347Z" 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:34.625 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "rgw", 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "", 2024-05-09T20:40:34.626 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:40:34.627 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:34.614326Z", 2024-05-09T20:40:34.627 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:40:34.627 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:40:34.627 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:34.627 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:34.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.628+0000 7f0d62fdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:34.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.632+0000 7f0eae55d980 20 remove_watcher() i=0 2024-05-09T20:40:34.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.632+0000 7f0eae55d980 2 removed watcher, disabling cache 2024-05-09T20:40:34.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.636+0000 7f0eae55d980 20 remove_watcher() i=1 2024-05-09T20:40:34.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.636+0000 7f0eae55d980 20 remove_watcher() i=2 2024-05-09T20:40:34.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.640+0000 7f0eae55d980 20 remove_watcher() i=3 2024-05-09T20:40:34.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.640+0000 7f0eae55d980 20 remove_watcher() i=4 2024-05-09T20:40:34.640 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.644+0000 7f0eae55d980 20 remove_watcher() i=5 2024-05-09T20:40:34.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.644+0000 7f0eae55d980 20 remove_watcher() i=6 2024-05-09T20:40:34.643 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.648+0000 7f0eae55d980 20 remove_watcher() i=7 2024-05-09T20:40:34.660 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 8065637661 --totp-seed GBOOB2FPCGHVOJXCPEBLY5PS7NJSDXALVBMOUCMCSK6I627QFDDR5PKHQYYGY5PE --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-09T20:40:34.750 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:34.750 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:34.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.796+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 realm 2024-05-09T20:40:34.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:34.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:34.797 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:34.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.800+0000 7f820ef9d980 20 searching for the correct realm 2024-05-09T20:40:34.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:34.804+0000 7f820cab1640 0 -- 172.21.15.63:0/2924243650 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f820803c950 msgr2=0x7f820801d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:35.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.020+0000 7f820cab1640 0 -- 172.21.15.63:0/2924243650 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f820803c950 msgr2=0x7f820801d400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:35.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:35.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:35.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:35.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:35.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:35.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:35.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:35.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:35.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 zone default found 2024-05-09T20:40:35.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 4 Realm: () 2024-05-09T20:40:35.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:35.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:35.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:35.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 zonegroup default 2024-05-09T20:40:35.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.032+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.036+0000 7f820ef9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.036+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.036+0000 7f820ef9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.036+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.036+0000 7f820ef9d980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.036+0000 7f820ef9d980 20 started sync module instance, tier type = 2024-05-09T20:40:35.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.036+0000 7f820ef9d980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:35.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 20 add_watcher() i=0 2024-05-09T20:40:35.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 20 add_watcher() i=1 2024-05-09T20:40:35.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 20 add_watcher() i=2 2024-05-09T20:40:35.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 20 add_watcher() i=3 2024-05-09T20:40:35.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 20 add_watcher() i=4 2024-05-09T20:40:35.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 20 add_watcher() i=5 2024-05-09T20:40:35.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 20 add_watcher() i=6 2024-05-09T20:40:35.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 20 add_watcher() i=7 2024-05-09T20:40:35.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.060+0000 7f820ef9d980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:35.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.064+0000 7f80c37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:35.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.064+0000 7f820ef9d980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:35.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.064+0000 7f820ef9d980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:35.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.064+0000 7f820ef9d980 5 note: GC not initialized 2024-05-09T20:40:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.064+0000 7f80c27e4640 20 reqs_thread_entry: start 2024-05-09T20:40:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.064+0000 7f820ef9d980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:35.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.068+0000 7f820ef9d980 20 Filter name: none 2024-05-09T20:40:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.068+0000 7f80bbfff640 20 reqs_thread_entry: start 2024-05-09T20:40:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.068+0000 7f820ef9d980 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2024-05-09T20:40:35.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.068+0000 7f820ef9d980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.068+0000 7f820ef9d980 20 rados_obj.operate() r=0 bl.length=519 2024-05-09T20:40:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.068+0000 7f820ef9d980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x13 2024-05-09T20:40:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.068+0000 7f820ef9d980 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-09T20:40:35.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.068+0000 7f820ef9d980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:35.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.072+0000 7f820ef9d980 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-05-09T20:40:35.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.072+0000 7f820ef9d980 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-09T20:40:35.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.072+0000 7f820ef9d980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser.client.0, ofs0, ns] 2024-05-09T20:40:35.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.072+0000 7f80e3fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333621 cookie 94029620736064 notifier 7725 bl.length()=733 2024-05-09T20:40:35.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.072+0000 7f80e3fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2024-05-09T20:40:35.070 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.072+0000 7f80e3fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2024-05-09T20:40:35.074 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.076+0000 7f80c37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:35.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.084+0000 7f820ef9d980 20 remove_watcher() i=0 2024-05-09T20:40:35.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.084+0000 7f820ef9d980 2 removed watcher, disabling cache 2024-05-09T20:40:35.082 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.084+0000 7f820ef9d980 20 remove_watcher() i=1 2024-05-09T20:40:35.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.088+0000 7f820ef9d980 20 remove_watcher() i=2 2024-05-09T20:40:35.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.088+0000 7f820ef9d980 20 remove_watcher() i=3 2024-05-09T20:40:35.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.092+0000 7f820ef9d980 20 remove_watcher() i=4 2024-05-09T20:40:35.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.092+0000 7f820ef9d980 20 remove_watcher() i=5 2024-05-09T20:40:35.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.096+0000 7f820ef9d980 20 remove_watcher() i=6 2024-05-09T20:40:35.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.096+0000 7f820ef9d980 20 remove_watcher() i=7 2024-05-09T20:40:35.110 DEBUG:tasks.s3tests:Creating user foobar.client.0 on client.0 2024-05-09T20:40:35.110 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foobar.client.0 --display-name Mr.foobar.client.0 --email foobar+test@test.test --access-key MITCRVAIFFINIVAIRQGJ --secret f4ffwwLE+6mvFRLWGwrpy45gpg1sSZkyJQZB7Jtrmrvk6U8nbDwIiw== --cluster ceph 2024-05-09T20:40:35.149 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:35.149 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:35.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 20 realm 2024-05-09T20:40:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.191 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.192+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.196+0000 7f90d92e9980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:35.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.196+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.196+0000 7f90d92e9980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:35.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.196+0000 7f90d92e9980 20 searching for the correct realm 2024-05-09T20:40:35.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.200+0000 7f90d75fe640 0 -- 172.21.15.63:0/1644979188 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f90cc02c070 msgr2=0x7f90cc02c960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:40:35.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.208+0000 7f90d65fc640 0 -- 172.21.15.63:0/1644979188 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f90c802f040 msgr2=0x7f90c801e3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:35.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:35.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:35.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:35.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:35.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:35.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:35.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.216+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 zone default found 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 4 Realm: () 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:35.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 zonegroup default 2024-05-09T20:40:35.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 started sync module instance, tier type = 2024-05-09T20:40:35.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.220+0000 7f90d92e9980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 20 add_watcher() i=0 2024-05-09T20:40:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 20 add_watcher() i=1 2024-05-09T20:40:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 20 add_watcher() i=2 2024-05-09T20:40:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 20 add_watcher() i=3 2024-05-09T20:40:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 20 add_watcher() i=4 2024-05-09T20:40:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 20 add_watcher() i=5 2024-05-09T20:40:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 20 add_watcher() i=6 2024-05-09T20:40:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 20 add_watcher() i=7 2024-05-09T20:40:35.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.236+0000 7f90d92e9980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f8f8d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 5 note: GC not initialized 2024-05-09T20:40:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f8f07fff640 20 reqs_thread_entry: start 2024-05-09T20:40:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 Filter name: none 2024-05-09T20:40:35.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f8f05ffb640 20 reqs_thread_entry: start 2024-05-09T20:40:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-05-09T20:40:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 cache get: name=default.rgw.meta+users.email+foobar+test@test.test : miss 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 adding default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 cache get: name=default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ : miss 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 cache put: name=default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ info.flags=0x0 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 adding default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ to cache LRU end 2024-05-09T20:40:35.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 cache get: name=default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ : hit (negative entry) 2024-05-09T20:40:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.240+0000 7f90d92e9980 10 cache get: name=default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ : hit (negative entry) 2024-05-09T20:40:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.244+0000 7f90d92e9980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-09T20:40:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.244+0000 7f90d92e9980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:35.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.244+0000 7f90d92e9980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-05-09T20:40:35.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.244+0000 7f8fba7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333521 cookie 93917508636416 notifier 7277 bl.length()=660 2024-05-09T20:40:35.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.244+0000 7f8fba7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-09T20:40:35.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.244+0000 7f8fba7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:35.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.248+0000 7f90d92e9980 10 cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-05-09T20:40:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.248+0000 7f90d92e9980 10 moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-05-09T20:40:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.248+0000 7f90d92e9980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.email:foobar+test@test.test, ofs0, ns] 2024-05-09T20:40:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.248+0000 7f8fba7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333572 cookie 93917508619056 notifier 7277 bl.length()=191 2024-05-09T20:40:35.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.248+0000 7f8fba7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foobar+test@test.test info.flags=0x7 2024-05-09T20:40:35.247 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.248+0000 7f8fba7fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+foobar+test@test.test to cache LRU end 2024-05-09T20:40:35.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.252+0000 7f90d92e9980 10 cache put: name=default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ info.flags=0x7 2024-05-09T20:40:35.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.252+0000 7f90d92e9980 10 moving default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ to cache LRU end 2024-05-09T20:40:35.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.252+0000 7f90d92e9980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:MITCRVAIFFINIVAIRQGJ, ofs0, ns] 2024-05-09T20:40:35.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.252+0000 7f8fb9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333573 cookie 93917508619056 notifier 7277 bl.length()=189 2024-05-09T20:40:35.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.252+0000 7f8fb9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ info.flags=0x7 2024-05-09T20:40:35.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.252+0000 7f8fb9ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MITCRVAIFFINIVAIRQGJ to cache LRU end 2024-05-09T20:40:35.253 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:35.253 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "foobar.client.0", 2024-05-09T20:40:35.253 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr.foobar.client.0", 2024-05-09T20:40:35.253 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "foobar+test@test.test", 2024-05-09T20:40:35.253 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:40:35.253 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:35.253 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "foobar.client.0", 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "MITCRVAIFFINIVAIRQGJ", 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "f4ffwwLE+6mvFRLWGwrpy45gpg1sSZkyJQZB7Jtrmrvk6U8nbDwIiw==", 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:35.245404Z" 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:40:35.254 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:35.255 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "rgw", 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "", 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:35.245389Z", 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:35.256 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:35.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.256+0000 7f8f8d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:35.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.260+0000 7f90d92e9980 20 remove_watcher() i=0 2024-05-09T20:40:35.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.260+0000 7f90d92e9980 2 removed watcher, disabling cache 2024-05-09T20:40:35.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.264+0000 7f90d92e9980 20 remove_watcher() i=1 2024-05-09T20:40:35.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.264+0000 7f90d92e9980 20 remove_watcher() i=2 2024-05-09T20:40:35.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.268+0000 7f90d92e9980 20 remove_watcher() i=3 2024-05-09T20:40:35.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.268+0000 7f90d92e9980 20 remove_watcher() i=4 2024-05-09T20:40:35.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.272+0000 7f90d92e9980 20 remove_watcher() i=5 2024-05-09T20:40:35.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.272+0000 7f90d92e9980 20 remove_watcher() i=6 2024-05-09T20:40:35.271 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.272+0000 7f90d92e9980 20 remove_watcher() i=7 2024-05-09T20:40:35.285 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foobar.client.0 --totp-serial 7468089079 --totp-seed BAOYPLXXJWDZCD2UIEL6ERJG42MOW5AVO73GG35RVAXYL2AQD5PG4DVHPFO7ZPQR --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-09T20:40:35.373 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:35.373 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:35.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 realm 2024-05-09T20:40:35.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:35.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:35.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.420+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.424+0000 7f4a7e482980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:35.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.424+0000 7f4a7e482980 20 searching for the correct realm 2024-05-09T20:40:35.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.424+0000 7f4a7b795640 0 -- 172.21.15.63:0/3297459526 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f4a6c0bab90 msgr2=0x7f4a6c0ac940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:40:35.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.636+0000 7f4a7b795640 0 -- 172.21.15.63:0/3297459526 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f4a6c0e5410 msgr2=0x7f4a6c0e5870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:35.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:35.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 zone default found 2024-05-09T20:40:35.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 4 Realm: () 2024-05-09T20:40:35.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:35.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:35.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:35.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 zonegroup default 2024-05-09T20:40:35.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.860+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.864+0000 7f4a7e482980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.864+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.864+0000 7f4a7e482980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.864+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.864+0000 7f4a7e482980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:35.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.864+0000 7f4a7e482980 20 started sync module instance, tier type = 2024-05-09T20:40:35.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.864+0000 7f4a7e482980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:35.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.884+0000 7f4a7e482980 20 add_watcher() i=0 2024-05-09T20:40:35.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.884+0000 7f4a7e482980 20 add_watcher() i=1 2024-05-09T20:40:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.884+0000 7f4a7e482980 20 add_watcher() i=2 2024-05-09T20:40:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.884+0000 7f4a7e482980 20 add_watcher() i=3 2024-05-09T20:40:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.884+0000 7f4a7e482980 20 add_watcher() i=4 2024-05-09T20:40:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.884+0000 7f4a7e482980 20 add_watcher() i=5 2024-05-09T20:40:35.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.884+0000 7f4a7e482980 20 add_watcher() i=6 2024-05-09T20:40:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.888+0000 7f4a7e482980 20 add_watcher() i=7 2024-05-09T20:40:35.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.888+0000 7f4a7e482980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:35.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f49327e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 5 note: GC not initialized 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f49317e2640 20 reqs_thread_entry: start 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 20 Filter name: none 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f492affd640 20 reqs_thread_entry: start 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-05-09T20:40:35.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 20 rados->read ofs=0 len=0 2024-05-09T20:40:35.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 20 rados_obj.operate() r=0 bl.length=472 2024-05-09T20:40:35.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-05-09T20:40:35.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:35.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.892+0000 7f4a7e482980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:35.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.900+0000 7f4a7e482980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-09T20:40:35.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.900+0000 7f4a7e482980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:35.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.900+0000 7f4a7e482980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-05-09T20:40:35.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.900+0000 7f495affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333522 cookie 94477044695392 notifier 7761 bl.length()=674 2024-05-09T20:40:35.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.900+0000 7f495affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-09T20:40:35.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.900+0000 7f495affd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:35.899 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.900+0000 7f49327e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:35.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.908+0000 7f4a7e482980 20 remove_watcher() i=0 2024-05-09T20:40:35.904 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.908+0000 7f4a7e482980 2 removed watcher, disabling cache 2024-05-09T20:40:35.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.908+0000 7f4a7e482980 20 remove_watcher() i=1 2024-05-09T20:40:35.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.912+0000 7f4a7e482980 20 remove_watcher() i=2 2024-05-09T20:40:35.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.912+0000 7f4a7e482980 20 remove_watcher() i=3 2024-05-09T20:40:35.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.916+0000 7f4a7e482980 20 remove_watcher() i=4 2024-05-09T20:40:35.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.916+0000 7f4a7e482980 20 remove_watcher() i=5 2024-05-09T20:40:35.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.920+0000 7f4a7e482980 20 remove_watcher() i=6 2024-05-09T20:40:35.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:35.920+0000 7f4a7e482980 20 remove_watcher() i=7 2024-05-09T20:40:35.933 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 caps add --uid foobar.client.0 --caps 'oidc-provider=*;roles=*;user-policy=*' --cluster ceph 2024-05-09T20:40:36.009 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:36.009 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:36.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 20 realm 2024-05-09T20:40:36.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:36.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.056+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.060+0000 7fc20252b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:36.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.060+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.060+0000 7fc20252b980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:36.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.060+0000 7fc20252b980 20 searching for the correct realm 2024-05-09T20:40:36.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.064+0000 7fc200840640 0 -- 172.21.15.63:0/2968384440 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7fc1f40c92a0 msgr2=0x7fc1f40d8040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:40:36.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.276+0000 7fc1ff83e640 0 -- 172.21.15.63:0/2968384440 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7fc1f0084290 msgr2=0x7fc1f0054fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:36.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.292+0000 7fc20252b980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:36.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.292+0000 7fc20252b980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:36.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.292+0000 7fc20252b980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:36.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.292+0000 7fc20252b980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:36.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.292+0000 7fc20252b980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:36.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.292+0000 7fc20252b980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:36.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.292+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:36.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:36.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 zone default found 2024-05-09T20:40:36.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 4 Realm: () 2024-05-09T20:40:36.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:36.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:36.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:36.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 zonegroup default 2024-05-09T20:40:36.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.296+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.300+0000 7fc20252b980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.300+0000 7fc20252b980 20 started sync module instance, tier type = 2024-05-09T20:40:36.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.300+0000 7fc20252b980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:36.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 20 add_watcher() i=0 2024-05-09T20:40:36.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 20 add_watcher() i=1 2024-05-09T20:40:36.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 20 add_watcher() i=2 2024-05-09T20:40:36.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 20 add_watcher() i=3 2024-05-09T20:40:36.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 20 add_watcher() i=4 2024-05-09T20:40:36.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 20 add_watcher() i=5 2024-05-09T20:40:36.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 20 add_watcher() i=6 2024-05-09T20:40:36.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 20 add_watcher() i=7 2024-05-09T20:40:36.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.324+0000 7fc20252b980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:36.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.328+0000 7fc0b77de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:36.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.328+0000 7fc20252b980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:36.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.328+0000 7fc20252b980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:36.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.328+0000 7fc20252b980 5 note: GC not initialized 2024-05-09T20:40:36.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc0b67dc640 20 reqs_thread_entry: start 2024-05-09T20:40:36.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:36.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 20 Filter name: none 2024-05-09T20:40:36.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc0af7fe640 20 reqs_thread_entry: start 2024-05-09T20:40:36.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 10 cache get: name=default.rgw.meta+users.uid+foobar.client.0 : miss 2024-05-09T20:40:36.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 20 rados_obj.operate() r=0 bl.length=486 2024-05-09T20:40:36.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x13 2024-05-09T20:40:36.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 10 adding default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:36.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:36.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 10 cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-09T20:40:36.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 10 moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:36.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.332+0000 7fc20252b980 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:foobar.client.0, ofs0, ns] 2024-05-09T20:40:36.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.336+0000 7fc0d77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333523 cookie 94386319787952 notifier 7310 bl.length()=727 2024-05-09T20:40:36.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.336+0000 7fc0d77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foobar.client.0 info.flags=0x17 2024-05-09T20:40:36.332 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.336+0000 7fc0d77fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foobar.client.0 to cache LRU end 2024-05-09T20:40:36.334 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:36.334 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "foobar.client.0", 2024-05-09T20:40:36.334 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr.foobar.client.0", 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "foobar+test@test.test", 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "foobar.client.0", 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "MITCRVAIFFINIVAIRQGJ", 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "f4ffwwLE+6mvFRLWGwrpy45gpg1sSZkyJQZB7Jtrmrvk6U8nbDwIiw==", 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:35.245404Z" 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:40:36.335 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [ 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "oidc-provider", 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: "perm": "*" 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "roles", 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: "perm": "*" 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "user-policy", 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: "perm": "*" 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:40:36.336 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:40:36.337 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "rgw", 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [ 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "7468089079" 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "", 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:40:36.338 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:35.245389Z", 2024-05-09T20:40:36.339 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:40:36.339 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:40:36.339 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:36.339 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:36.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.340+0000 7fc0b77de640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.344+0000 7fc20252b980 20 remove_watcher() i=0 2024-05-09T20:40:36.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.344+0000 7fc20252b980 2 removed watcher, disabling cache 2024-05-09T20:40:36.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.348+0000 7fc20252b980 20 remove_watcher() i=1 2024-05-09T20:40:36.346 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.348+0000 7fc20252b980 20 remove_watcher() i=2 2024-05-09T20:40:36.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.352+0000 7fc20252b980 20 remove_watcher() i=3 2024-05-09T20:40:36.350 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.352+0000 7fc20252b980 20 remove_watcher() i=4 2024-05-09T20:40:36.352 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.356+0000 7fc20252b980 20 remove_watcher() i=5 2024-05-09T20:40:36.354 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.356+0000 7fc20252b980 20 remove_watcher() i=6 2024-05-09T20:40:36.356 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.360+0000 7fc20252b980 20 remove_watcher() i=7 2024-05-09T20:40:36.374 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW88888888888888888 --account-name Mr.RGW88888888888888888 --email RGW88888888888888888+test@test.test 2024-05-09T20:40:36.455 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:36.456 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:36.499 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.500+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 realm 2024-05-09T20:40:36.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:36.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:36.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.504+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.508+0000 7ff93fa79980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:36.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.508+0000 7ff93fa79980 20 searching for the correct realm 2024-05-09T20:40:36.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.512+0000 7ff93d58d640 0 -- 172.21.15.63:0/1413529652 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7ff93801a040 msgr2=0x7ff938019920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:36.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.728+0000 7ff93d58d640 0 -- 172.21.15.63:0/1413529652 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7ff93801a040 msgr2=0x7ff938019920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:36.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:36.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:36.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:36.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:36.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:36.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:36.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:36.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:36.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 zone default found 2024-05-09T20:40:36.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 4 Realm: () 2024-05-09T20:40:36.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:36.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:36.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:36.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 zonegroup default 2024-05-09T20:40:36.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.740+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.744+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.744+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.744+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.744+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.744+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.744+0000 7ff93fa79980 20 started sync module instance, tier type = 2024-05-09T20:40:36.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.744+0000 7ff93fa79980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:36.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.768+0000 7ff93fa79980 20 add_watcher() i=0 2024-05-09T20:40:36.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.768+0000 7ff93fa79980 20 add_watcher() i=1 2024-05-09T20:40:36.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.768+0000 7ff93fa79980 20 add_watcher() i=2 2024-05-09T20:40:36.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.768+0000 7ff93fa79980 20 add_watcher() i=3 2024-05-09T20:40:36.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.768+0000 7ff93fa79980 20 add_watcher() i=4 2024-05-09T20:40:36.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.768+0000 7ff93fa79980 20 add_watcher() i=5 2024-05-09T20:40:36.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.768+0000 7ff93fa79980 20 add_watcher() i=6 2024-05-09T20:40:36.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.772+0000 7ff93fa79980 20 add_watcher() i=7 2024-05-09T20:40:36.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.772+0000 7ff93fa79980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:36.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff7f3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:36.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:36.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:36.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 5 note: GC not initialized 2024-05-09T20:40:36.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff7f2fe5640 20 reqs_thread_entry: start 2024-05-09T20:40:36.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:36.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 Filter name: none 2024-05-09T20:40:36.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff7f0fe1640 20 reqs_thread_entry: start 2024-05-09T20:40:36.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 : miss 2024-05-09T20:40:36.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x0 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 10 adding default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 failed to read name.$Mr.RGW88888888888888888 with: (2) No such file or directory 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 10 cache get: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test : miss 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x0 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 10 adding default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-05-09T20:40:36.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.776+0000 7ff93fa79980 20 failed to read rgw88888888888888888+test@test.test with: (2) No such file or directory 2024-05-09T20:40:36.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.780+0000 7ff93fa79980 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-05-09T20:40:36.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.780+0000 7ff93fa79980 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-05-09T20:40:36.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.780+0000 7ff93fa79980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW88888888888888888, ofs0, ns] 2024-05-09T20:40:36.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.780+0000 7ff820ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333574 cookie 94853432204672 notifier 7328 bl.length()=384 2024-05-09T20:40:36.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.780+0000 7ff820ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-05-09T20:40:36.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.780+0000 7ff820ff9640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-05-09T20:40:36.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.788+0000 7ff93fa79980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-05-09T20:40:36.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.788+0000 7ff93fa79980 10 moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-05-09T20:40:36.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.788+0000 7ff93fa79980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW88888888888888888, ofs0, ns] 2024-05-09T20:40:36.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.788+0000 7ff820ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333579 cookie 94853432212896 notifier 7328 bl.length()=225 2024-05-09T20:40:36.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.788+0000 7ff820ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 info.flags=0x17 2024-05-09T20:40:36.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.788+0000 7ff820ff9640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW88888888888888888 to cache LRU end 2024-05-09T20:40:36.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.792+0000 7ff93fa79980 10 cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-05-09T20:40:36.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.792+0000 7ff93fa79980 10 moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-05-09T20:40:36.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.792+0000 7ff93fa79980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw88888888888888888+test@test.test, ofs0, ns] 2024-05-09T20:40:36.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.792+0000 7ff80ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333535 cookie 94853432210768 notifier 7328 bl.length()=234 2024-05-09T20:40:36.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.792+0000 7ff80ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw88888888888888888+test@test.test info.flags=0x17 2024-05-09T20:40:36.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.792+0000 7ff80ffff640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw88888888888888888+test@test.test to cache LRU end 2024-05-09T20:40:36.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.796+0000 7ff7f3fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:36.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.804+0000 7ff93fa79980 20 remove_watcher() i=0 2024-05-09T20:40:36.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.804+0000 7ff93fa79980 2 removed watcher, disabling cache 2024-05-09T20:40:36.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.804+0000 7ff93fa79980 20 remove_watcher() i=1 2024-05-09T20:40:36.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.808+0000 7ff93fa79980 20 remove_watcher() i=2 2024-05-09T20:40:36.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.808+0000 7ff93fa79980 20 remove_watcher() i=3 2024-05-09T20:40:36.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.812+0000 7ff93fa79980 20 remove_watcher() i=4 2024-05-09T20:40:36.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.812+0000 7ff93fa79980 20 remove_watcher() i=5 2024-05-09T20:40:36.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.816+0000 7ff93fa79980 20 remove_watcher() i=6 2024-05-09T20:40:36.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.816+0000 7ff93fa79980 20 remove_watcher() i=7 2024-05-09T20:40:36.827 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:36.827 INFO:teuthology.orchestra.run.smithi063.stdout: "id": "RGW88888888888888888", 2024-05-09T20:40:36.827 INFO:teuthology.orchestra.run.smithi063.stdout: "tenant": "", 2024-05-09T20:40:36.827 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "Mr.RGW88888888888888888", 2024-05-09T20:40:36.827 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "RGW88888888888888888+test@test.test", 2024-05-09T20:40:36.827 INFO:teuthology.orchestra.run.smithi063.stdout: "quota": { 2024-05-09T20:40:36.827 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:36.827 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:36.828 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:36.828 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:36.828 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:36.828 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:36.828 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:36.828 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:36.828 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max_users": 1000, 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max_roles": 1000, 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max_groups": 1000, 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout: "max_access_keys": 4 2024-05-09T20:40:36.829 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:36.830 DEBUG:tasks.s3tests:Creating user RGW88888888888888888 on client.0 2024-05-09T20:40:36.831 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root1.client.0 --display-name Mr.RGW88888888888888888 --email root1+test@test.test --access-key PQVLJQNGHOVQEVZJOGIW --secret LZE6AnSF+FcNRcoTr1FzyxFT4SqZFa395b6X50WsUOPprrTuP7NvaA== --cluster ceph --account-id RGW88888888888888888 --account-root 2024-05-09T20:40:36.919 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:36.919 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:36.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.964+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.964+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.964+0000 7f2dd3564980 20 realm 2024-05-09T20:40:36.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.964+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.964+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.968+0000 7f2dd3564980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:36.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.968+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.968+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:36.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.968+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.968+0000 7f2dd3564980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:36.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.968+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:36.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.968+0000 7f2dd3564980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:36.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.968+0000 7f2dd3564980 20 searching for the correct realm 2024-05-09T20:40:36.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:36.972+0000 7f2dd1879640 0 -- 172.21.15.63:0/153801773 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f2dc8008050 msgr2=0x7f2dc8077770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:37.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.184+0000 7f2dd0877640 0 -- 172.21.15.63:0/153801773 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f2dc40c1070 msgr2=0x7f2dc40c3b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:37.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:37.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:37.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:37.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:37.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:37.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:37.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:37.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:37.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 zone default found 2024-05-09T20:40:37.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 4 Realm: () 2024-05-09T20:40:37.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:37.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:37.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:37.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 zonegroup default 2024-05-09T20:40:37.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.204+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.208+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.208+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.208+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.208+0000 7f2dd3564980 20 started sync module instance, tier type = 2024-05-09T20:40:37.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.208+0000 7f2dd3564980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:37.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.228+0000 7f2dd3564980 20 add_watcher() i=0 2024-05-09T20:40:37.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.228+0000 7f2dd3564980 20 add_watcher() i=1 2024-05-09T20:40:37.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.228+0000 7f2dd3564980 20 add_watcher() i=2 2024-05-09T20:40:37.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.228+0000 7f2dd3564980 20 add_watcher() i=3 2024-05-09T20:40:37.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.228+0000 7f2dd3564980 20 add_watcher() i=4 2024-05-09T20:40:37.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.228+0000 7f2dd3564980 20 add_watcher() i=5 2024-05-09T20:40:37.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.228+0000 7f2dd3564980 20 add_watcher() i=6 2024-05-09T20:40:37.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.232+0000 7f2dd3564980 20 add_watcher() i=7 2024-05-09T20:40:37.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.232+0000 7f2dd3564980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:37.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2c877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:37.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:37.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:37.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 5 note: GC not initialized 2024-05-09T20:40:37.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2c867e4640 20 reqs_thread_entry: start 2024-05-09T20:40:37.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:37.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 Filter name: none 2024-05-09T20:40:37.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2bf7fff640 20 reqs_thread_entry: start 2024-05-09T20:40:37.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-05-09T20:40:37.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x0 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 cache get: name=default.rgw.meta+users.email+root1+test@test.test : miss 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x0 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 adding default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 cache get: name=default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW : miss 2024-05-09T20:40:37.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 cache put: name=default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW info.flags=0x0 2024-05-09T20:40:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 adding default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW to cache LRU end 2024-05-09T20:40:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 10 cache get: name=default.rgw.meta+accounts+account.RGW88888888888888888 : miss 2024-05-09T20:40:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.236+0000 7f2dd3564980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.240+0000 7f2dd3564980 20 rados_obj.operate() r=0 bl.length=184 2024-05-09T20:40:37.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.240+0000 7f2dd3564980 10 cache put: name=default.rgw.meta+accounts+account.RGW88888888888888888 info.flags=0x17 2024-05-09T20:40:37.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.240+0000 7f2dd3564980 10 adding default.rgw.meta+accounts+account.RGW88888888888888888 to cache LRU end 2024-05-09T20:40:37.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.240+0000 7f2dd3564980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:37.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.240+0000 7f2dd3564980 10 cache get: name=default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW : hit (negative entry) 2024-05-09T20:40:37.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.240+0000 7f2dd3564980 10 cache get: name=default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW : hit (negative entry) 2024-05-09T20:40:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.244+0000 7f2dd3564980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-05-09T20:40:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.244+0000 7f2dd3564980 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-09T20:40:37.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.244+0000 7f2dd3564980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-05-09T20:40:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.244+0000 7f2cb4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333575 cookie 94816381468160 notifier 7812 bl.length()=681 2024-05-09T20:40:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.244+0000 7f2cb4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-05-09T20:40:37.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.244+0000 7f2cb4ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-09T20:40:37.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.248+0000 7f2dd3564980 10 cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-05-09T20:40:37.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.248+0000 7f2dd3564980 10 moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-05-09T20:40:37.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.248+0000 7f2dd3564980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root1+test@test.test, ofs0, ns] 2024-05-09T20:40:37.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.248+0000 7f2cb4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333622 cookie 94816381475376 notifier 7812 bl.length()=189 2024-05-09T20:40:37.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.248+0000 7f2cb4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root1+test@test.test info.flags=0x7 2024-05-09T20:40:37.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.248+0000 7f2cb4ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+root1+test@test.test to cache LRU end 2024-05-09T20:40:37.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.252+0000 7f2dd3564980 10 cache put: name=default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW info.flags=0x7 2024-05-09T20:40:37.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.252+0000 7f2dd3564980 10 moving default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW to cache LRU end 2024-05-09T20:40:37.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.252+0000 7f2dd3564980 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.keys:PQVLJQNGHOVQEVZJOGIW, ofs0, ns] 2024-05-09T20:40:37.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.252+0000 7f2cb4ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333580 cookie 94816381461984 notifier 7812 bl.length()=188 2024-05-09T20:40:37.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.252+0000 7f2cb4ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW info.flags=0x7 2024-05-09T20:40:37.250 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.252+0000 7f2cb4ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+PQVLJQNGHOVQEVZJOGIW to cache LRU end 2024-05-09T20:40:37.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.256+0000 7f2dd3564980 20 linked user root1.client.0 to account RGW88888888888888888 2024-05-09T20:40:37.255 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:37.255 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "root1.client.0", 2024-05-09T20:40:37.255 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr.RGW88888888888888888", 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "root1+test@test.test", 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "root1.client.0", 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "PQVLJQNGHOVQEVZJOGIW", 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "LZE6AnSF+FcNRcoTr1FzyxFT4SqZFa395b6X50WsUOPprrTuP7NvaA==", 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:37.243211Z" 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:40:37.256 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:37.257 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "root", 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "RGW88888888888888888", 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:37.243172Z", 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:37.258 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:37.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.260+0000 7f2c877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:37.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.264+0000 7f2dd3564980 20 remove_watcher() i=0 2024-05-09T20:40:37.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.264+0000 7f2dd3564980 2 removed watcher, disabling cache 2024-05-09T20:40:37.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.268+0000 7f2dd3564980 20 remove_watcher() i=1 2024-05-09T20:40:37.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.268+0000 7f2dd3564980 20 remove_watcher() i=2 2024-05-09T20:40:37.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.272+0000 7f2dd3564980 20 remove_watcher() i=3 2024-05-09T20:40:37.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.272+0000 7f2dd3564980 20 remove_watcher() i=4 2024-05-09T20:40:37.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.276+0000 7f2dd3564980 20 remove_watcher() i=5 2024-05-09T20:40:37.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.276+0000 7f2dd3564980 20 remove_watcher() i=6 2024-05-09T20:40:37.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.276+0000 7f2dd3564980 20 remove_watcher() i=7 2024-05-09T20:40:37.290 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root1.client.0 --totp-serial 9321926363 --totp-seed AVQHUDAE7HAYOLCYYPOQWMT6VNDM6DEC5L4FUPIQK5PJSHVAADRPZ4LIG4KKOLTW --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-09T20:40:37.337 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:37.337 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:37.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.380+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 realm 2024-05-09T20:40:37.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:37.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.381 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:37.382 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.384+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.388+0000 7f619f430980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:37.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.388+0000 7f619f430980 20 searching for the correct realm 2024-05-09T20:40:37.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.388+0000 7f619c743640 0 -- 172.21.15.63:0/1450509561 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f618c008070 msgr2=0x7f618c05d340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:37.598 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.600+0000 7f619c743640 0 -- 172.21.15.63:0/1450509561 >> [v2:172.21.15.120:6802/610448732,v1:172.21.15.120:6807/610448732] conn(0x7f618c087040 msgr2=0x7f618c062120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:40:37.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:37.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:37.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:37.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:37.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:37.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:37.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:37.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:37.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 zone default found 2024-05-09T20:40:37.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 4 Realm: () 2024-05-09T20:40:37.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:37.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:37.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:37.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 zonegroup default 2024-05-09T20:40:37.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.824+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.828+0000 7f619f430980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.828+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.828+0000 7f619f430980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.828+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.828+0000 7f619f430980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.828+0000 7f619f430980 20 started sync module instance, tier type = 2024-05-09T20:40:37.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.828+0000 7f619f430980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.848+0000 7f619f430980 20 add_watcher() i=0 2024-05-09T20:40:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.848+0000 7f619f430980 20 add_watcher() i=1 2024-05-09T20:40:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.848+0000 7f619f430980 20 add_watcher() i=2 2024-05-09T20:40:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.848+0000 7f619f430980 20 add_watcher() i=3 2024-05-09T20:40:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.848+0000 7f619f430980 20 add_watcher() i=4 2024-05-09T20:40:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.848+0000 7f619f430980 20 add_watcher() i=5 2024-05-09T20:40:37.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.848+0000 7f619f430980 20 add_watcher() i=6 2024-05-09T20:40:37.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.852+0000 7f619f430980 20 add_watcher() i=7 2024-05-09T20:40:37.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.852+0000 7f619f430980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:37.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f60537e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:37.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f619f430980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:37.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f619f430980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:37.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f619f430980 5 note: GC not initialized 2024-05-09T20:40:37.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f60527e4640 20 reqs_thread_entry: start 2024-05-09T20:40:37.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f619f430980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:37.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f619f430980 20 Filter name: none 2024-05-09T20:40:37.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f604bfff640 20 reqs_thread_entry: start 2024-05-09T20:40:37.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f619f430980 10 cache get: name=default.rgw.meta+users.uid+root1.client.0 : miss 2024-05-09T20:40:37.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.856+0000 7f619f430980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.860+0000 7f619f430980 20 rados_obj.operate() r=0 bl.length=494 2024-05-09T20:40:37.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.860+0000 7f619f430980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x13 2024-05-09T20:40:37.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.860+0000 7f619f430980 10 adding default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-09T20:40:37.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.860+0000 7f619f430980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:37.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.864+0000 7f619f430980 10 cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-05-09T20:40:37.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.864+0000 7f619f430980 10 moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-09T20:40:37.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.864+0000 7f619f430980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root1.client.0, ofs0, ns] 2024-05-09T20:40:37.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.864+0000 7f6073fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333576 cookie 94741181573232 notifier 7358 bl.length()=695 2024-05-09T20:40:37.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.864+0000 7f6073fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root1.client.0 info.flags=0x17 2024-05-09T20:40:37.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.864+0000 7f6073fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root1.client.0 to cache LRU end 2024-05-09T20:40:37.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.872+0000 7f60537e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:37.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.876+0000 7f619f430980 20 remove_watcher() i=0 2024-05-09T20:40:37.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.876+0000 7f619f430980 2 removed watcher, disabling cache 2024-05-09T20:40:37.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.876+0000 7f619f430980 20 remove_watcher() i=1 2024-05-09T20:40:37.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.880+0000 7f619f430980 20 remove_watcher() i=2 2024-05-09T20:40:37.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.884+0000 7f619f430980 20 remove_watcher() i=3 2024-05-09T20:40:37.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.884+0000 7f619f430980 20 remove_watcher() i=4 2024-05-09T20:40:37.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.888+0000 7f619f430980 20 remove_watcher() i=5 2024-05-09T20:40:37.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.888+0000 7f619f430980 20 remove_watcher() i=6 2024-05-09T20:40:37.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.892+0000 7f619f430980 20 remove_watcher() i=7 2024-05-09T20:40:37.906 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph account create --account-id RGW99999999999999999 --account-name Mr.RGW99999999999999999 --email RGW99999999999999999+test@test.test 2024-05-09T20:40:37.946 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:37.946 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:37.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.988+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 realm 2024-05-09T20:40:37.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:37.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:37.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:37.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:37.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:37.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.992+0000 7fb22531c980 20 searching for the correct realm 2024-05-09T20:40:37.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:37.996+0000 7fb222e30640 0 -- 172.21.15.63:0/1691564977 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fb21c0d8020 msgr2=0x7fb21c0fd630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-05-09T20:40:38.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.008+0000 7fb222e30640 0 -- 172.21.15.63:0/1691564977 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7fb0ec0781f0 msgr2=0x7fb0ec07a620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:38.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.012+0000 7fb22531c980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:38.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.012+0000 7fb22531c980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:38.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.012+0000 7fb22531c980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:38.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.012+0000 7fb22531c980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:38.011 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.012+0000 7fb22531c980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:38.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.012+0000 7fb22531c980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:38.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.012+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 zone default found 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 4 Realm: () 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 zonegroup default 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 started sync module instance, tier type = 2024-05-09T20:40:38.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.016+0000 7fb22531c980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:38.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 20 add_watcher() i=0 2024-05-09T20:40:38.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 20 add_watcher() i=1 2024-05-09T20:40:38.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 20 add_watcher() i=2 2024-05-09T20:40:38.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 20 add_watcher() i=3 2024-05-09T20:40:38.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 20 add_watcher() i=4 2024-05-09T20:40:38.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 20 add_watcher() i=5 2024-05-09T20:40:38.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 20 add_watcher() i=6 2024-05-09T20:40:38.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 20 add_watcher() i=7 2024-05-09T20:40:38.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.032+0000 7fb22531c980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:38.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb0d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:38.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 5 note: GC not initialized 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb0d8fe1640 20 reqs_thread_entry: start 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 20 Filter name: none 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb0d1ffb640 20 reqs_thread_entry: start 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 10 cache get: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 : miss 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x0 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 10 adding default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 20 failed to read name.$Mr.RGW99999999999999999 with: (2) No such file or directory 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 10 cache get: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test : miss 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.036+0000 7fb22531c980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb22531c980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb22531c980 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x0 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb22531c980 10 adding default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-05-09T20:40:38.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb22531c980 20 failed to read rgw99999999999999999+test@test.test with: (2) No such file or directory 2024-05-09T20:40:38.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb22531c980 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-05-09T20:40:38.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb22531c980 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-05-09T20:40:38.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb22531c980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:accounts:account.RGW99999999999999999, ofs0, ns] 2024-05-09T20:40:38.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb1067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333577 cookie 94719637757680 notifier 7373 bl.length()=384 2024-05-09T20:40:38.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb1067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-05-09T20:40:38.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.040+0000 7fb1067fc640 10 rgw watcher librados: moving default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-05-09T20:40:38.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.044+0000 7fb22531c980 10 cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-05-09T20:40:38.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.044+0000 7fb22531c980 10 moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-05-09T20:40:38.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.044+0000 7fb22531c980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:accounts:name.$Mr.RGW99999999999999999, ofs0, ns] 2024-05-09T20:40:38.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.048+0000 7fb105ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333581 cookie 94719637766784 notifier 7373 bl.length()=225 2024-05-09T20:40:38.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.048+0000 7fb105ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 info.flags=0x17 2024-05-09T20:40:38.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.048+0000 7fb105ffb640 10 rgw watcher librados: moving default.rgw.meta+accounts+name.$Mr.RGW99999999999999999 to cache LRU end 2024-05-09T20:40:38.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.052+0000 7fb22531c980 10 cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-05-09T20:40:38.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.052+0000 7fb22531c980 10 moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-05-09T20:40:38.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.052+0000 7fb22531c980 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:rgw99999999999999999+test@test.test, ofs0, ns] 2024-05-09T20:40:38.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.052+0000 7fb1067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333536 cookie 94719637764288 notifier 7373 bl.length()=234 2024-05-09T20:40:38.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.052+0000 7fb1067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+rgw99999999999999999+test@test.test info.flags=0x17 2024-05-09T20:40:38.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.052+0000 7fb1067fc640 10 rgw watcher librados: moving default.rgw.meta+users.email+rgw99999999999999999+test@test.test to cache LRU end 2024-05-09T20:40:38.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.056+0000 7fb0d9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:38.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.060+0000 7fb22531c980 20 remove_watcher() i=0 2024-05-09T20:40:38.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.060+0000 7fb22531c980 2 removed watcher, disabling cache 2024-05-09T20:40:38.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.060+0000 7fb22531c980 20 remove_watcher() i=1 2024-05-09T20:40:38.061 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.064+0000 7fb22531c980 20 remove_watcher() i=2 2024-05-09T20:40:38.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.064+0000 7fb22531c980 20 remove_watcher() i=3 2024-05-09T20:40:38.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.068+0000 7fb22531c980 20 remove_watcher() i=4 2024-05-09T20:40:38.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.068+0000 7fb22531c980 20 remove_watcher() i=5 2024-05-09T20:40:38.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.068+0000 7fb22531c980 20 remove_watcher() i=6 2024-05-09T20:40:38.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.072+0000 7fb22531c980 20 remove_watcher() i=7 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "id": "RGW99999999999999999", 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "tenant": "", 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "name": "Mr.RGW99999999999999999", 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "RGW99999999999999999+test@test.test", 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "quota": { 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:38.080 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max_users": 1000, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max_roles": 1000, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max_groups": 1000, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout: "max_access_keys": 4 2024-05-09T20:40:38.081 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:38.082 DEBUG:tasks.s3tests:Creating user RGW99999999999999999 on client.0 2024-05-09T20:40:38.083 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid root2.client.0 --display-name Mr.RGW99999999999999999 --email root2+test@test.test --access-key WEAJYAIHVDCUWYZAMZYA --secret C/B4lvMFhmeGZh1GR7ajoDs/cll3Fd6oBPQ7SI7fdD1eXDAKVv/y6A== --cluster ceph --account-id RGW99999999999999999 --account-root 2024-05-09T20:40:38.165 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:38.165 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:38.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.208+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.208+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.208+0000 7f2a88839980 20 realm 2024-05-09T20:40:38.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.208+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.208+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.208+0000 7f2a88839980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:38.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.208+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.212+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.212+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.212+0000 7f2a88839980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:38.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.212+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.212+0000 7f2a88839980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:38.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.212+0000 7f2a88839980 20 searching for the correct realm 2024-05-09T20:40:38.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.216+0000 7f2a86b4e640 0 -- 172.21.15.63:0/4037431083 >> [v2:172.21.15.63:6816/1087089498,v1:172.21.15.63:6817/1087089498] conn(0x7f2a70008050 msgr2=0x7f2a700551e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:38.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.224+0000 7f2a85b4c640 0 -- 172.21.15.63:0/4037431083 >> [v2:172.21.15.120:6800/3964176807,v1:172.21.15.120:6805/3964176807] conn(0x7f2a781005d0 msgr2=0x7f2a780f8850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.232+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 zone default found 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 4 Realm: () 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 zonegroup default 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 started sync module instance, tier type = 2024-05-09T20:40:38.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.236+0000 7f2a88839980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:38.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 20 add_watcher() i=0 2024-05-09T20:40:38.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 20 add_watcher() i=1 2024-05-09T20:40:38.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 20 add_watcher() i=2 2024-05-09T20:40:38.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 20 add_watcher() i=3 2024-05-09T20:40:38.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 20 add_watcher() i=4 2024-05-09T20:40:38.248 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 20 add_watcher() i=5 2024-05-09T20:40:38.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 20 add_watcher() i=6 2024-05-09T20:40:38.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 20 add_watcher() i=7 2024-05-09T20:40:38.249 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.252+0000 7f2a88839980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:38.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f293cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 5 note: GC not initialized 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f28b77fe640 20 reqs_thread_entry: start 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 Filter name: none 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f28b57fa640 20 reqs_thread_entry: start 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-05-09T20:40:38.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x0 2024-05-09T20:40:38.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-09T20:40:38.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 cache get: name=default.rgw.meta+users.email+root2+test@test.test : miss 2024-05-09T20:40:38.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x0 2024-05-09T20:40:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 adding default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-05-09T20:40:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 cache get: name=default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA : miss 2024-05-09T20:40:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 cache put: name=default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA info.flags=0x0 2024-05-09T20:40:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 adding default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA to cache LRU end 2024-05-09T20:40:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 cache get: name=default.rgw.meta+accounts+account.RGW99999999999999999 : miss 2024-05-09T20:40:38.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 20 rados_obj.operate() r=0 bl.length=184 2024-05-09T20:40:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 cache put: name=default.rgw.meta+accounts+account.RGW99999999999999999 info.flags=0x17 2024-05-09T20:40:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 adding default.rgw.meta+accounts+account.RGW99999999999999999 to cache LRU end 2024-05-09T20:40:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.256+0000 7f2a88839980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.260+0000 7f2a88839980 10 cache get: name=default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA : hit (negative entry) 2024-05-09T20:40:38.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.260+0000 7f2a88839980 10 cache get: name=default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA : hit (negative entry) 2024-05-09T20:40:38.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.260+0000 7f2a88839980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-05-09T20:40:38.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.260+0000 7f2a88839980 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-09T20:40:38.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.260+0000 7f2a88839980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-05-09T20:40:38.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.260+0000 7f2969ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333578 cookie 94744940050160 notifier 7869 bl.length()=681 2024-05-09T20:40:38.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.260+0000 7f2969ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-05-09T20:40:38.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.260+0000 7f2969ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-09T20:40:38.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.268+0000 7f2a88839980 10 cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-05-09T20:40:38.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.268+0000 7f2a88839980 10 moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-05-09T20:40:38.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.268+0000 7f2a88839980 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.email:root2+test@test.test, ofs0, ns] 2024-05-09T20:40:38.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.268+0000 7f29697fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333623 cookie 94744940062192 notifier 7869 bl.length()=189 2024-05-09T20:40:38.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.268+0000 7f29697fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+root2+test@test.test info.flags=0x7 2024-05-09T20:40:38.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.268+0000 7f29697fa640 10 rgw watcher librados: moving default.rgw.meta+users.email+root2+test@test.test to cache LRU end 2024-05-09T20:40:38.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.272+0000 7f2a88839980 10 cache put: name=default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA info.flags=0x7 2024-05-09T20:40:38.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.272+0000 7f2a88839980 10 moving default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA to cache LRU end 2024-05-09T20:40:38.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.272+0000 7f2a88839980 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.keys:WEAJYAIHVDCUWYZAMZYA, ofs0, ns] 2024-05-09T20:40:38.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.272+0000 7f29697fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333582 cookie 94744940052496 notifier 7869 bl.length()=188 2024-05-09T20:40:38.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.272+0000 7f29697fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA info.flags=0x7 2024-05-09T20:40:38.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.272+0000 7f29697fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+WEAJYAIHVDCUWYZAMZYA to cache LRU end 2024-05-09T20:40:38.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.276+0000 7f2a88839980 20 linked user root2.client.0 to account RGW99999999999999999 2024-05-09T20:40:38.273 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-05-09T20:40:38.273 INFO:teuthology.orchestra.run.smithi063.stdout: "user_id": "root2.client.0", 2024-05-09T20:40:38.273 INFO:teuthology.orchestra.run.smithi063.stdout: "display_name": "Mr.RGW99999999999999999", 2024-05-09T20:40:38.273 INFO:teuthology.orchestra.run.smithi063.stdout: "email": "root2+test@test.test", 2024-05-09T20:40:38.273 INFO:teuthology.orchestra.run.smithi063.stdout: "suspended": 0, 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "max_buckets": 1000, 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "subusers": [], 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "keys": [ 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: { 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "user": "root2.client.0", 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "access_key": "WEAJYAIHVDCUWYZAMZYA", 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "secret_key": "C/B4lvMFhmeGZh1GR7ajoDs/cll3Fd6oBPQ7SI7fdD1eXDAKVv/y6A==", 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "active": true, 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:38.261973Z" 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "swift_keys": [], 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "caps": [], 2024-05-09T20:40:38.274 INFO:teuthology.orchestra.run.smithi063.stdout: "op_mask": "read, write, delete", 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "default_placement": "", 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "default_storage_class": "", 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "placement_tags": [], 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "bucket_quota": { 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "user_quota": { 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "enabled": false, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "check_on_raw": false, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size": -1, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max_size_kb": 0, 2024-05-09T20:40:38.275 INFO:teuthology.orchestra.run.smithi063.stdout: "max_objects": -1 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: "temp_url_keys": [], 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: "type": "root", 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: "mfa_ids": [], 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: "account_id": "RGW99999999999999999", 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: "path": "/", 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: "create_date": "2024-05-09T20:40:38.261956Z", 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: "tags": [], 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: "group_ids": [] 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-05-09T20:40:38.276 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:40:38.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.276+0000 7f293cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:38.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.284+0000 7f2a88839980 20 remove_watcher() i=0 2024-05-09T20:40:38.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.284+0000 7f2a88839980 2 removed watcher, disabling cache 2024-05-09T20:40:38.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.284+0000 7f2a88839980 20 remove_watcher() i=1 2024-05-09T20:40:38.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.288+0000 7f2a88839980 20 remove_watcher() i=2 2024-05-09T20:40:38.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.288+0000 7f2a88839980 20 remove_watcher() i=3 2024-05-09T20:40:38.287 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.292+0000 7f2a88839980 20 remove_watcher() i=4 2024-05-09T20:40:38.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.292+0000 7f2a88839980 20 remove_watcher() i=5 2024-05-09T20:40:38.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.292+0000 7f2a88839980 20 remove_watcher() i=6 2024-05-09T20:40:38.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.296+0000 7f2a88839980 20 remove_watcher() i=7 2024-05-09T20:40:38.306 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid root2.client.0 --totp-serial 2137996931 --totp-seed T63AKDNUB6YFYVP4BYCWCW7JCTX5P3TCX5WYJ4T4T3HSKS7CLDPW7YYGM3X3BWJ4 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2024-05-09T20:40:38.388 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T20:40:38.388 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T20:40:38.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 20 realm 2024-05-09T20:40:38.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-09T20:40:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.432+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.436+0000 7f29d3338980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:38.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.436+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.436+0000 7f29d3338980 20 rados_obj.operate() r=0 bl.length=1139 2024-05-09T20:40:38.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.436+0000 7f29d3338980 20 searching for the correct realm 2024-05-09T20:40:38.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.440+0000 7f29d0e4c640 0 -- 172.21.15.63:0/3338715535 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f29c81242d0 msgr2=0x7f29c8124a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:38.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.656+0000 7f29d0e4c640 0 -- 172.21.15.63:0/3338715535 >> [v2:172.21.15.120:6801/982132549,v1:172.21.15.120:6803/982132549] conn(0x7f29c81242d0 msgr2=0x7f29c8124a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-05-09T20:40:38.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 RGWRados::pool_iterate: got zonegroup_info.1fe76006-ec22-466c-887e-e78d5475ac56 2024-05-09T20:40:38.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-09T20:40:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 RGWRados::pool_iterate: got default.zone. 2024-05-09T20:40:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 RGWRados::pool_iterate: got zone_info.6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 2024-05-09T20:40:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 RGWRados::pool_iterate: got zone_names.default 2024-05-09T20:40:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-09T20:40:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.666 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 rados_obj.operate() r=0 bl.length=46 2024-05-09T20:40:38.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.668+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 rados_obj.operate() r=0 bl.length=470 2024-05-09T20:40:38.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 zone default found 2024-05-09T20:40:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 4 Realm: () 2024-05-09T20:40:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 4 ZoneGroup: default (1fe76006-ec22-466c-887e-e78d5475ac56) 2024-05-09T20:40:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 4 Zone: default (6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0) 2024-05-09T20:40:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-09T20:40:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 zonegroup default 2024-05-09T20:40:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 rados_obj.operate() r=-2 bl.length=0 2024-05-09T20:40:38.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 started sync module instance, tier type = 2024-05-09T20:40:38.670 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.672+0000 7f29d3338980 20 started zone id=6d736ab1-252c-4ec6-8c6e-7c5d07ae5bf0 (name=default) with tier type = 2024-05-09T20:40:38.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.696+0000 7f29d3338980 20 add_watcher() i=0 2024-05-09T20:40:38.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.696+0000 7f29d3338980 20 add_watcher() i=1 2024-05-09T20:40:38.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.696+0000 7f29d3338980 20 add_watcher() i=2 2024-05-09T20:40:38.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.696+0000 7f29d3338980 20 add_watcher() i=3 2024-05-09T20:40:38.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.696+0000 7f29d3338980 20 add_watcher() i=4 2024-05-09T20:40:38.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.696+0000 7f29d3338980 20 add_watcher() i=5 2024-05-09T20:40:38.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.696+0000 7f29d3338980 20 add_watcher() i=6 2024-05-09T20:40:38.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.700+0000 7f29d3338980 20 add_watcher() i=7 2024-05-09T20:40:38.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.700+0000 7f29d3338980 2 all 8 watchers are set, enabling cache 2024-05-09T20:40:38.701 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f28877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:38.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f29d3338980 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-09T20:40:38.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f29d3338980 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-09T20:40:38.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f29d3338980 5 note: GC not initialized 2024-05-09T20:40:38.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f28867e4640 20 reqs_thread_entry: start 2024-05-09T20:40:38.702 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f29d3338980 20 init_complete bucket index max shards: 11 2024-05-09T20:40:38.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f29d3338980 20 Filter name: none 2024-05-09T20:40:38.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f287f7fe640 20 reqs_thread_entry: start 2024-05-09T20:40:38.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f29d3338980 10 cache get: name=default.rgw.meta+users.uid+root2.client.0 : miss 2024-05-09T20:40:38.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.704+0000 7f29d3338980 20 rados->read ofs=0 len=0 2024-05-09T20:40:38.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.708+0000 7f29d3338980 20 rados_obj.operate() r=0 bl.length=494 2024-05-09T20:40:38.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.708+0000 7f29d3338980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x13 2024-05-09T20:40:38.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.708+0000 7f29d3338980 10 adding default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-09T20:40:38.704 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.708+0000 7f29d3338980 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-09T20:40:38.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.712+0000 7f29d3338980 10 cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-05-09T20:40:38.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.712+0000 7f29d3338980 10 moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-09T20:40:38.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.712+0000 7f29d3338980 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.uid:root2.client.0, ofs0, ns] 2024-05-09T20:40:38.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.712+0000 7f28a7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 1138166333579 cookie 93832660010848 notifier 7887 bl.length()=695 2024-05-09T20:40:38.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.712+0000 7f28a7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+root2.client.0 info.flags=0x17 2024-05-09T20:40:38.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.712+0000 7f28a7fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+root2.client.0 to cache LRU end 2024-05-09T20:40:38.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.716+0000 7f28877e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-09T20:40:38.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.724+0000 7f29d3338980 20 remove_watcher() i=0 2024-05-09T20:40:38.720 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.724+0000 7f29d3338980 2 removed watcher, disabling cache 2024-05-09T20:40:38.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.724+0000 7f29d3338980 20 remove_watcher() i=1 2024-05-09T20:40:38.725 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.728+0000 7f29d3338980 20 remove_watcher() i=2 2024-05-09T20:40:38.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.732+0000 7f29d3338980 20 remove_watcher() i=3 2024-05-09T20:40:38.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.732+0000 7f29d3338980 20 remove_watcher() i=4 2024-05-09T20:40:38.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.736+0000 7f29d3338980 20 remove_watcher() i=5 2024-05-09T20:40:38.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.736+0000 7f29d3338980 20 remove_watcher() i=6 2024-05-09T20:40:38.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-05-09T20:40:38.736+0000 7f29d3338980 20 remove_watcher() i=7 2024-05-09T20:40:38.751 INFO:tasks.s3tests:Configuring s3-tests... 2024-05-09T20:40:38.752 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:40:38.752 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2024-05-09T20:40:38.801 INFO:tasks.s3tests:Configuring boto... 2024-05-09T20:40:38.801 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-05-09T20:40:38.801 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/boto-client.0.cfg 2024-05-09T20:40:38.848 DEBUG:teuthology.orchestra.run.smithi063:s3 tests against rgw> source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-05-09T20:40:39.013 INFO:teuthology.orchestra.run.smithi063.stdout:GLOB sdist-make: /home/ubuntu/cephtest/s3-tests-client.0/setup.py 2024-05-09T20:40:39.216 INFO:teuthology.orchestra.run.smithi063.stdout:py create: /home/ubuntu/cephtest/s3-tests-client.0/.tox/py 2024-05-09T20:40:39.497 INFO:teuthology.orchestra.run.smithi063.stdout:py installdeps: -rrequirements.txt 2024-05-09T20:42:32.282 INFO:teuthology.orchestra.run.smithi063.stdout:py inst: /home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip 2024-05-09T20:42:34.484 INFO:teuthology.orchestra.run.smithi063.stdout:py installed: boto==2.49.0,boto3==1.24.96,botocore==1.27.96,cachetools==5.3.3,certifi==2024.2.2,chardet==5.2.0,charset-normalizer==3.3.2,colorama==0.4.6,distlib==0.3.8,exceptiongroup==1.2.1,filelock==3.14.0,gevent==24.2.1,greenlet==3.0.3,httplib2==0.22.0,idna==3.7,iniconfig==2.0.0,isodate==0.6.1,jmespath==1.0.1,lxml==5.2.1,munch==4.0.0,packaging==24.0,platformdirs==4.2.1,pluggy==1.5.0,pyparsing==3.1.2,pyproject-api==1.6.1,pytest==8.2.0,python-dateutil==2.9.0.post0,pytz==2024.1,PyYAML==6.0.1,requests==2.31.0,s3tests @ file:///home/ubuntu/cephtest/s3-tests-client.0/.tox/.tmp/package/1/s3tests-0.0.1.zip#sha256=974b4edb3785ccc9a382fea1aec2fd2ed4b8c96b57505fb3b22576c2552446ca,s3transfer==0.6.2,six==1.16.0,tomli==2.0.1,tox==4.15.0,urllib3==1.26.18,virtualenv==20.26.1,zope.event==5.0,zope.interface==6.3 2024-05-09T20:42:34.484 INFO:teuthology.orchestra.run.smithi063.stdout:py run-test-pre: PYTHONHASHSEED='2062751602' 2024-05-09T20:42:34.484 INFO:teuthology.orchestra.run.smithi063.stdout:py run-test: commands[0] | pytest -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3' 2024-05-09T20:42:34.694 INFO:teuthology.orchestra.run.smithi063.stdout:============================= test session starts ============================== 2024-05-09T20:42:34.694 INFO:teuthology.orchestra.run.smithi063.stdout:platform linux -- Python 3.10.6, pytest-8.2.0, pluggy-1.5.0 -- /home/ubuntu/cephtest/s3-tests-client.0/.tox/py/bin/python 2024-05-09T20:42:34.694 INFO:teuthology.orchestra.run.smithi063.stdout:cachedir: .tox/py/.pytest_cache 2024-05-09T20:42:34.695 INFO:teuthology.orchestra.run.smithi063.stdout:rootdir: /home/ubuntu/cephtest/s3-tests-client.0 2024-05-09T20:42:34.695 INFO:teuthology.orchestra.run.smithi063.stdout:configfile: pytest.ini 2024-05-09T20:42:36.455 INFO:teuthology.orchestra.run.smithi063.stdout:collecting ... collected 859 items / 127 deselected / 732 selected 2024-05-09T20:42:36.455 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-05-09T20:42:37.366 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_none PASSED [ 0%] 2024-05-09T20:42:38.159 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_empty PASSED [ 0%] 2024-05-09T20:42:39.020 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_date_and_amz_date PASSED [ 0%] 2024-05-09T20:42:39.905 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_amz_date_and_no_date PASSED [ 0%] 2024-05-09T20:42:40.671 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_none PASSED [ 0%] 2024-05-09T20:42:41.401 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_contentlength_none PASSED [ 0%] 2024-05-09T20:42:42.421 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_acl_create_contentlength_none PASSED [ 0%] 2024-05-09T20:42:43.139 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_contentlength_none PASSED [ 1%] 2024-05-09T20:42:43.313 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_empty PASSED [ 1%] 2024-05-09T20:42:43.484 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_none PASSED [ 1%] 2024-05-09T20:42:44.265 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws2 PASSED [ 1%] 2024-05-09T20:42:45.039 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws2 PASSED [ 1%] 2024-05-09T20:42:45.787 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws2 PASSED [ 1%] 2024-05-09T20:42:46.540 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws2 PASSED [ 1%] 2024-05-09T20:42:46.720 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws2 PASSED [ 2%] 2024-05-09T20:42:46.904 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws2 PASSED [ 2%] 2024-05-09T20:42:47.027 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws4 SKIPPED [ 2%] 2024-05-09T20:42:47.146 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_contentlength_mismatch_below_aws4 SKIPPED [ 2%] 2024-05-09T20:42:47.268 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_incorrect_aws4 SKIPPED [ 2%] 2024-05-09T20:42:47.389 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_authorization_invalid_aws4 SKIPPED [ 2%] 2024-05-09T20:42:47.512 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_empty_aws4 SKIPPED [ 2%] 2024-05-09T20:42:47.632 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_ua_none_aws4 SKIPPED [ 3%] 2024-05-09T20:42:47.753 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_invalid_aws4 SKIPPED [ 3%] 2024-05-09T20:42:47.879 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_invalid_aws4 SKIPPED [ 3%] 2024-05-09T20:42:48.002 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_empty_aws4 SKIPPED [ 3%] 2024-05-09T20:42:48.122 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_empty_aws4 SKIPPED [ 3%] 2024-05-09T20:42:48.242 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_none_aws4 SKIPPED [ 3%] 2024-05-09T20:42:48.361 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_none_aws4 SKIPPED [ 3%] 2024-05-09T20:42:48.486 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_today_aws4 SKIPPED [ 3%] 2024-05-09T20:42:48.607 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_today_aws4 SKIPPED [ 4%] 2024-05-09T20:42:48.727 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_today_aws4 SKIPPED [ 4%] 2024-05-09T20:42:48.853 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_today_aws4 SKIPPED [ 4%] 2024-05-09T20:42:48.973 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws4 SKIPPED [ 4%] 2024-05-09T20:42:49.096 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 4%] 2024-05-09T20:42:49.238 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_date_after_end_aws4 SKIPPED [ 4%] 2024-05-09T20:42:49.373 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_bad_amz_date_after_end_aws4 SKIPPED [ 4%] 2024-05-09T20:42:49.494 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_custom_header_aws4 SKIPPED [ 5%] 2024-05-09T20:42:49.615 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_object_create_missing_signed_header_aws4 SKIPPED [ 5%] 2024-05-09T20:42:49.759 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_authorization_invalid_aws4 SKIPPED [ 5%] 2024-05-09T20:42:49.885 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws4 SKIPPED [ 5%] 2024-05-09T20:42:50.008 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_ua_none_aws4 SKIPPED [ 5%] 2024-05-09T20:42:50.128 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws4 SKIPPED [ 5%] 2024-05-09T20:42:50.248 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_invalid_aws4 SKIPPED [ 5%] 2024-05-09T20:42:50.369 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_empty_aws4 SKIPPED [ 6%] 2024-05-09T20:42:50.491 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_empty_aws4 SKIPPED [ 6%] 2024-05-09T20:42:50.612 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_none_aws4 SKIPPED [ 6%] 2024-05-09T20:42:50.738 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_none_aws4 SKIPPED [ 6%] 2024-05-09T20:42:50.860 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws4 SKIPPED [ 6%] 2024-05-09T20:42:50.980 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_today_aws4 SKIPPED [ 6%] 2024-05-09T20:42:51.100 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws4 SKIPPED [ 6%] 2024-05-09T20:42:51.219 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_after_today_aws4 SKIPPED [ 6%] 2024-05-09T20:42:51.338 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws4 SKIPPED [ 7%] 2024-05-09T20:42:51.458 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_headers.py::test_bucket_create_bad_amz_date_before_epoch_aws4 SKIPPED [ 7%] 2024-05-09T20:42:52.783 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_versioning_obj_read_not_exist_null PASSED [ 7%] 2024-05-09T20:42:58.125 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_object_storage_class PASSED [ 7%] 2024-05-09T20:43:05.527 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_object_storage_class_multipart PASSED [ 7%] 2024-05-09T20:43:16.523 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class PASSED [ 7%] 2024-05-09T20:43:29.865 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_object_modify_storage_class_multipart PASSED [ 7%] 2024-05-09T20:43:35.189 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy PASSED [ 8%] 2024-05-09T20:43:40.583 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_object_storage_class_copy_multipart PASSED [ 8%] 2024-05-09T20:43:42.401 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_atomic_dual_conditional_write_1mb PASSED [ 8%] 2024-05-09T20:43:43.087 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_atomic_write_bucket_gone PASSED [ 8%] 2024-05-09T20:43:44.643 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_1 PASSED [ 8%] 2024-05-09T20:43:46.373 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_encryption_sse_c_multipart_invalid_chunks_2 PASSED [ 8%] 2024-05-09T20:43:48.588 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_bucket_policy_set_condition_operator_end_with_IfExists PASSED [ 8%] 2024-05-09T20:43:50.053 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3.py::test_bucket_policy_put_obj_request_obj_tag PASSED [ 9%] 2024-05-09T20:43:51.154 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_nonexistant_bucket_rgw SKIPPED [ 9%] 2024-05-09T20:43:51.159 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_public_index SKIPPED [ 9%] 2024-05-09T20:43:51.161 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_public_index SKIPPED [ 9%] 2024-05-09T20:43:51.163 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty SKIPPED [ 9%] 2024-05-09T20:43:51.166 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty SKIPPED [ 9%] 2024-05-09T20:43:51.169 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index SKIPPED [ 9%] 2024-05-09T20:43:51.171 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index SKIPPED [ 9%] 2024-05-09T20:43:51.174 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-05-09T20:43:51.176 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_missingerrordoc SKIPPED [ 10%] 2024-05-09T20:43:51.179 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-05-09T20:43:51.182 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_missingerrordoc SKIPPED [ 10%] 2024-05-09T20:43:51.185 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-05-09T20:43:51.187 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_pubilc_errordoc SKIPPED [ 10%] 2024-05-09T20:43:51.190 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_blockederrordoc SKIPPED [ 10%] 2024-05-09T20:43:51.192 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-05-09T20:43:51.195 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_blockederrordoc SKIPPED [ 11%] 2024-05-09T20:43:51.198 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-05-09T20:43:51.201 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_empty_gooderrordoc SKIPPED [ 11%] 2024-05-09T20:43:51.203 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_public_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-05-09T20:43:51.207 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_private_bucket_list_private_index_gooderrordoc SKIPPED [ 11%] 2024-05-09T20:43:51.246 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_base SKIPPED [ 11%] 2024-05-09T20:43:51.247 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path SKIPPED [ 12%] 2024-05-09T20:43:51.249 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_bucket_private_redirectall_path_upgrade SKIPPED [ 12%] 2024-05-09T20:43:51.250 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_nonwebsite SKIPPED [ 12%] 2024-05-09T20:43:51.252 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_relative SKIPPED [ 12%] 2024-05-09T20:43:51.253 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_public_abs SKIPPED [ 12%] 2024-05-09T20:43:51.254 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_relative SKIPPED [ 12%] 2024-05-09T20:43:51.255 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_website_xredirect_private_abs SKIPPED [ 12%] 2024-05-09T20:43:51.352 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_s3_website.py::test_routing_generator XFAIL [ 12%] 2024-05-09T20:43:51.361 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests/functional/test_utils.py::test_generate PASSED [ 13%] 2024-05-09T20:43:53.630 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_short PASSED [ 13%] 2024-05-09T20:44:05.741 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_bad PASSED [ 13%] 2024-05-09T20:44:07.061 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_empty PASSED [ 13%] 2024-05-09T20:44:08.573 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_none PASSED [ 13%] 2024-05-09T20:44:10.090 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_mismatch PASSED [ 13%] 2024-05-09T20:44:11.585 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_empty PASSED [ 13%] 2024-05-09T20:44:13.069 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_expect_none PASSED [ 14%] 2024-05-09T20:44:14.273 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contentlength_negative PASSED [ 14%] 2024-05-09T20:44:15.797 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_invalid PASSED [ 14%] 2024-05-09T20:44:17.165 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_empty PASSED [ 14%] 2024-05-09T20:44:18.593 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_contenttype_none PASSED [ 14%] 2024-05-09T20:44:19.838 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_put_bad_canned_acl PASSED [ 14%] 2024-05-09T20:44:21.057 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_mismatch PASSED [ 14%] 2024-05-09T20:44:22.249 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_expect_empty PASSED [ 15%] 2024-05-09T20:44:22.797 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_contentlength_negative PASSED [ 15%] 2024-05-09T20:44:24.070 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_md5_invalid_garbage_aws2 PASSED [ 15%] 2024-05-09T20:44:25.566 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_empty_aws2 PASSED [ 15%] 2024-05-09T20:44:27.009 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_ua_none_aws2 PASSED [ 15%] 2024-05-09T20:44:28.226 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_invalid_aws2 PASSED [ 15%] 2024-05-09T20:44:29.486 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_empty_aws2 PASSED [ 15%] 2024-05-09T20:44:30.754 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_today_aws2 PASSED [ 15%] 2024-05-09T20:44:31.965 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_before_epoch_aws2 PASSED [ 16%] 2024-05-09T20:44:33.181 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_object_create_bad_date_after_end_aws2 PASSED [ 16%] 2024-05-09T20:44:34.334 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_empty_aws2 PASSED [ 16%] 2024-05-09T20:44:35.494 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_ua_none_aws2 PASSED [ 16%] 2024-05-09T20:44:36.102 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_invalid_aws2 PASSED [ 16%] 2024-05-09T20:44:36.682 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_empty_aws2 PASSED [ 16%] 2024-05-09T20:44:37.282 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_today_aws2 PASSED [ 16%] 2024-05-09T20:44:37.898 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_after_today_aws2 PASSED [ 17%] 2024-05-09T20:44:38.478 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_headers.py::test_bucket_create_bad_date_before_epoch_aws2 PASSED [ 17%] 2024-05-09T20:44:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy PASSED [ 17%] 2024-05-09T20:44:40.258 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_invalid_user PASSED [ 17%] 2024-05-09T20:44:40.946 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_put_user_policy_parameter_limit PASSED [ 17%] 2024-05-09T20:44:41.938 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_put_existing_user_policy PASSED [ 17%] 2024-05-09T20:44:42.854 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy PASSED [ 17%] 2024-05-09T20:44:43.474 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_list_user_policy_invalid_user PASSED [ 18%] 2024-05-09T20:44:44.406 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy PASSED [ 18%] 2024-05-09T20:44:45.422 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_invalid_user PASSED [ 18%] 2024-05-09T20:44:46.686 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_get_user_policy_from_multiple_policies PASSED [ 18%] 2024-05-09T20:44:47.518 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy PASSED [ 18%] 2024-05-09T20:44:48.434 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_user PASSED [ 18%] 2024-05-09T20:44:49.358 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_invalid_policy_name PASSED [ 18%] 2024-05-09T20:44:50.958 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_delete_user_policy_from_multiple_policies PASSED [ 18%] 2024-05-09T20:44:52.618 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_allow_bucket_actions_in_user_policy PASSED [ 19%] 2024-05-09T20:44:54.078 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_deny_bucket_actions_in_user_policy PASSED [ 19%] 2024-05-09T20:44:55.862 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_allow_object_actions_in_user_policy PASSED [ 19%] 2024-05-09T20:44:57.846 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_deny_object_actions_in_user_policy PASSED [ 19%] 2024-05-09T20:45:00.582 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_allow_multipart_actions_in_user_policy PASSED [ 19%] 2024-05-09T20:45:03.258 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_deny_multipart_actions_in_user_policy PASSED [ 19%] 2024-05-09T20:45:05.266 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_allow_tagging_actions_in_user_policy PASSED [ 19%] 2024-05-09T20:45:07.250 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_deny_tagging_actions_in_user_policy PASSED [ 20%] 2024-05-09T20:45:08.734 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policy_statements PASSED [ 20%] 2024-05-09T20:45:10.562 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_verify_conflicting_user_policies PASSED [ 20%] 2024-05-09T20:45:11.658 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_verify_allow_iam_actions PASSED [ 20%] 2024-05-09T20:45:14.311 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_create PASSED [ 20%] 2024-05-09T20:45:15.934 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_case_insensitive_name PASSED [ 20%] 2024-05-09T20:45:17.907 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_delete PASSED [ 20%] 2024-05-09T20:45:21.811 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list PASSED [ 21%] 2024-05-09T20:45:26.935 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_list_path_prefix PASSED [ 21%] 2024-05-09T20:45:29.847 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_name PASSED [ 21%] 2024-05-09T20:45:32.587 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_update_path PASSED [ 21%] 2024-05-09T20:45:34.642 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_create PASSED [ 21%] 2024-05-09T20:45:36.046 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_create PASSED [ 21%] 2024-05-09T20:45:38.514 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_update PASSED [ 21%] 2024-05-09T20:45:40.239 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_update PASSED [ 21%] 2024-05-09T20:45:42.563 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_delete PASSED [ 22%] 2024-05-09T20:45:44.143 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_current_user_access_key_delete PASSED [ 22%] 2024-05-09T20:45:47.135 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_access_key_list PASSED [ 22%] 2024-05-09T20:45:49.811 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_bucket_policy_allow PASSED [ 22%] 2024-05-09T20:45:53.111 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy PASSED [ 22%] 2024-05-09T20:45:56.451 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_managed PASSED [ 22%] 2024-05-09T20:45:58.731 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_user_policy_allow PASSED [ 22%] 2024-05-09T20:46:00.859 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_create PASSED [ 23%] 2024-05-09T20:46:02.539 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_case_insensitive_name PASSED [ 23%] 2024-05-09T20:46:06.563 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_list PASSED [ 23%] 2024-05-09T20:46:09.511 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_group_update PASSED [ 23%] 2024-05-09T20:46:12.799 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy PASSED [ 23%] 2024-05-09T20:46:15.903 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy PASSED [ 23%] 2024-05-09T20:46:18.943 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_inline_group_policy_allow PASSED [ 23%] 2024-05-09T20:46:21.919 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_managed_group_policy_allow PASSED [ 24%] 2024-05-09T20:46:24.535 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_create PASSED [ 24%] 2024-05-09T20:46:26.299 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_case_insensitive_name PASSED [ 24%] 2024-05-09T20:46:28.335 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_delete PASSED [ 24%] 2024-05-09T20:46:32.056 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list PASSED [ 24%] 2024-05-09T20:46:36.828 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_list_path_prefix PASSED [ 24%] 2024-05-09T20:46:38.840 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_update PASSED [ 24%] 2024-05-09T20:46:41.955 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy PASSED [ 25%] 2024-05-09T20:46:44.980 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_managed PASSED [ 25%] 2024-05-09T20:46:47.987 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow PASSED [ 25%] 2024-05-09T20:46:51.124 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_same_account_role_policy_allow PASSED [ 25%] 2024-05-09T20:46:54.495 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_role_policy_allow PASSED [ 25%] 2024-05-09T20:46:57.695 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_create_bucket PASSED [ 25%] 2024-05-09T20:47:00.727 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_role_policy_allow_get_role PASSED [ 25%] 2024-05-09T20:47:02.795 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_account_oidc_provider PASSED [ 25%] 2024-05-09T20:47:05.948 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_user_arn PASSED [ 26%] 2024-05-09T20:47:09.156 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_user_arn PASSED [ 26%] 2024-05-09T20:47:12.304 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_arn PASSED [ 26%] 2024-05-09T20:47:15.452 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_arn PASSED [ 26%] 2024-05-09T20:47:18.692 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_bucket_policy_allow_account_id PASSED [ 26%] 2024-05-09T20:47:21.804 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_user_policy_allow_account_id PASSED [ 26%] 2024-05-09T20:47:25.024 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_user_id PASSED [ 26%] 2024-05-09T20:47:28.172 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_user_id PASSED [ 27%] 2024-05-09T20:47:31.344 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_canonical_id PASSED [ 27%] 2024-05-09T20:47:34.500 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_canonical_id PASSED [ 27%] 2024-05-09T20:47:37.660 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_bucket_acl_user_policy_grant_account_email PASSED [ 27%] 2024-05-09T20:47:40.876 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_user_policy_bucket_acl_grant_account_email PASSED [ 27%] 2024-05-09T20:47:43.012 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_arn PASSED [ 27%] 2024-05-09T20:47:45.120 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_policy_allow_account_id PASSED [ 27%] 2024-05-09T20:47:47.264 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_canonical_id PASSED [ 28%] 2024-05-09T20:47:49.300 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_iam.py::test_cross_account_root_bucket_acl_grant_account_email PASSED [ 28%] 2024-05-09T20:47:50.864 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_empty PASSED [ 28%] 2024-05-09T20:47:53.015 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_distinct PASSED [ 28%] 2024-05-09T20:47:54.840 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_many PASSED [ 28%] 2024-05-09T20:47:56.699 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_many PASSED [ 28%] 2024-05-09T20:47:58.672 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_basic_key_count PASSED [ 28%] 2024-05-09T20:48:00.543 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_basic PASSED [ 28%] 2024-05-09T20:48:02.428 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_basic PASSED [ 29%] 2024-05-09T20:48:04.255 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_encoding_basic PASSED [ 29%] 2024-05-09T20:48:06.130 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_encoding_basic PASSED [ 29%] 2024-05-09T20:48:08.851 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix PASSED [ 29%] 2024-05-09T20:48:11.603 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix PASSED [ 29%] 2024-05-09T20:48:13.116 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-05-09T20:48:14.592 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_ends_with_delimiter PASSED [ 29%] 2024-05-09T20:48:16.415 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_alt PASSED [ 30%] 2024-05-09T20:48:18.271 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_alt PASSED [ 30%] 2024-05-09T20:48:20.947 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_prefix_underscore PASSED [ 30%] 2024-05-09T20:48:23.707 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_prefix_underscore PASSED [ 30%] 2024-05-09T20:48:25.519 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_percentage PASSED [ 30%] 2024-05-09T20:48:27.367 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_percentage PASSED [ 30%] 2024-05-09T20:48:29.200 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_whitespace PASSED [ 30%] 2024-05-09T20:48:31.036 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_whitespace PASSED [ 31%] 2024-05-09T20:48:32.940 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_dot PASSED [ 31%] 2024-05-09T20:48:34.747 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_dot PASSED [ 31%] 2024-05-09T20:48:36.587 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_unreadable PASSED [ 31%] 2024-05-09T20:48:38.456 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_unreadable PASSED [ 31%] 2024-05-09T20:48:40.284 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_empty PASSED [ 31%] 2024-05-09T20:48:42.111 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_empty PASSED [ 31%] 2024-05-09T20:48:43.980 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_none PASSED [ 31%] 2024-05-09T20:48:45.892 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_none PASSED [ 32%] 2024-05-09T20:48:47.620 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_notempty PASSED [ 32%] 2024-05-09T20:48:49.340 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_defaultempty PASSED [ 32%] 2024-05-09T20:48:51.076 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_fetchowner_empty PASSED [ 32%] 2024-05-09T20:48:52.952 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_exist PASSED [ 32%] 2024-05-09T20:48:54.788 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_delimiter_not_exist PASSED [ 32%] 2024-05-09T20:50:58.625 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_delimiter_not_skip_special PASSED [ 32%] 2024-05-09T20:51:00.312 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_basic PASSED [ 33%] 2024-05-09T20:51:02.069 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_basic PASSED [ 33%] 2024-05-09T20:51:03.812 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_alt PASSED [ 33%] 2024-05-09T20:51:05.516 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_alt PASSED [ 33%] 2024-05-09T20:51:07.289 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_empty PASSED [ 33%] 2024-05-09T20:51:09.008 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_empty PASSED [ 33%] 2024-05-09T20:51:10.712 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_none PASSED [ 33%] 2024-05-09T20:51:12.481 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_none PASSED [ 34%] 2024-05-09T20:51:14.209 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_not_exist PASSED [ 34%] 2024-05-09T20:51:16.000 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_not_exist PASSED [ 34%] 2024-05-09T20:51:17.744 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_unreadable PASSED [ 34%] 2024-05-09T20:51:19.448 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_unreadable PASSED [ 34%] 2024-05-09T20:51:21.257 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_basic PASSED [ 34%] 2024-05-09T20:51:23.133 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_basic PASSED [ 34%] 2024-05-09T20:51:24.953 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_alt PASSED [ 34%] 2024-05-09T20:51:26.786 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_alt PASSED [ 35%] 2024-05-09T20:51:28.660 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-05-09T20:51:30.476 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_not_exist PASSED [ 35%] 2024-05-09T20:51:32.377 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-05-09T20:51:34.221 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_delimiter_not_exist PASSED [ 35%] 2024-05-09T20:51:36.061 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-05-09T20:51:37.873 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist PASSED [ 35%] 2024-05-09T20:51:39.880 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_one PASSED [ 36%] 2024-05-09T20:51:41.852 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_one PASSED [ 36%] 2024-05-09T20:51:43.725 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_zero PASSED [ 36%] 2024-05-09T20:51:45.560 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_zero PASSED [ 36%] 2024-05-09T20:51:47.461 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_none PASSED [ 36%] 2024-05-09T20:51:49.321 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_maxkeys_none PASSED [ 36%] 2024-05-09T20:51:49.996 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_account_usage PASSED [ 36%] 2024-05-09T20:51:51.465 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_head_bucket_usage PASSED [ 37%] 2024-05-09T20:51:56.537 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_unordered PASSED [ 37%] 2024-05-09T20:52:01.517 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_unordered PASSED [ 37%] 2024-05-09T20:52:03.357 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_maxkeys_invalid PASSED [ 37%] 2024-05-09T20:52:05.188 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_none PASSED [ 37%] 2024-05-09T20:52:07.105 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_empty PASSED [ 37%] 2024-05-09T20:52:08.997 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken_empty PASSED [ 37%] 2024-05-09T20:52:10.977 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_continuationtoken PASSED [ 37%] 2024-05-09T20:52:13.001 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_both_continuationtoken_startafter PASSED [ 38%] 2024-05-09T20:52:15.057 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_unreadable PASSED [ 38%] 2024-05-09T20:52:16.901 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_unreadable PASSED [ 38%] 2024-05-09T20:52:18.769 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_not_in_list PASSED [ 38%] 2024-05-09T20:52:20.657 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_not_in_list PASSED [ 38%] 2024-05-09T20:52:22.557 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_marker_after_list PASSED [ 38%] 2024-05-09T20:52:24.401 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_startafter_after_list PASSED [ 38%] 2024-05-09T20:52:26.665 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data PASSED [ 39%] 2024-05-09T20:52:29.845 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_return_data_versioning PASSED [ 39%] 2024-05-09T20:52:31.173 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous PASSED [ 39%] 2024-05-09T20:52:32.593 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous PASSED [ 39%] 2024-05-09T20:52:33.801 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_objects_anonymous_fail PASSED [ 39%] 2024-05-09T20:52:35.021 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_listv2_objects_anonymous_fail PASSED [ 39%] 2024-05-09T20:52:35.645 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_notexist PASSED [ 39%] 2024-05-09T20:52:36.333 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucketv2_notexist PASSED [ 40%] 2024-05-09T20:52:37.093 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_notexist PASSED [ 40%] 2024-05-09T20:52:38.649 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_delete_nonempty PASSED [ 40%] 2024-05-09T20:52:43.357 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_concurrent_set_canned_acl PASSED [ 40%] 2024-05-09T20:52:43.973 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_to_nonexist_bucket PASSED [ 40%] 2024-05-09T20:52:45.445 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_with_chunked_transfer_encoding PASSED [ 40%] 2024-05-09T20:52:46.605 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_delete PASSED [ 40%] 2024-05-09T20:52:47.905 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_read_not_exist PASSED [ 40%] 2024-05-09T20:52:49.149 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_requestid_matches_header_on_error PASSED [ 41%] 2024-05-09T20:52:54.157 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_versioning_concurrent_multi_object_delete PASSED [ 41%] 2024-05-09T20:52:56.221 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete PASSED [ 41%] 2024-05-09T20:52:58.333 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete PASSED [ 41%] 2024-05-09T20:55:01.158 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multi_object_delete_key_limit PASSED [ 41%] 2024-05-09T20:57:04.011 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multi_objectv2_delete_key_limit PASSED [ 41%] 2024-05-09T20:57:05.454 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_head_zero_bytes PASSED [ 41%] 2024-05-09T20:57:06.878 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_check_etag PASSED [ 42%] 2024-05-09T20:57:08.326 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_cache_control PASSED [ 42%] 2024-05-09T20:57:09.802 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_expires PASSED [ 42%] 2024-05-09T20:57:11.522 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_read_update_read_delete PASSED [ 42%] 2024-05-09T20:57:12.994 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_good PASSED [ 42%] 2024-05-09T20:57:14.462 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_none_to_empty PASSED [ 42%] 2024-05-09T20:57:16.146 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_set_get_metadata_overwrite_to_empty PASSED [ 42%] 2024-05-09T20:57:17.754 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_metadata_replaced_on_put PASSED [ 43%] 2024-05-09T20:57:19.274 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_write_file PASSED [ 43%] 2024-05-09T20:57:20.818 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_anonymous_request PASSED [ 43%] 2024-05-09T20:57:22.454 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request PASSED [ 43%] 2024-05-09T20:57:23.934 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_no_content_type PASSED [ 43%] 2024-05-09T20:57:25.190 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_authenticated_request_bad_access_key PASSED [ 43%] 2024-05-09T20:57:26.594 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_success_code PASSED [ 43%] 2024-05-09T20:57:27.962 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_invalid_success_code PASSED [ 43%] 2024-05-09T20:57:29.822 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_larger_than_chunk PASSED [ 44%] 2024-05-09T20:57:31.342 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_set_key_from_filename PASSED [ 44%] 2024-05-09T20:57:32.799 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_ignored_header PASSED [ 44%] 2024-05-09T20:57:34.166 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_case_insensitive_condition_fields PASSED [ 44%] 2024-05-09T20:57:35.650 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_escaped_field_values PASSED [ 44%] 2024-05-09T20:57:37.250 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_success_redirect_action PASSED [ 44%] 2024-05-09T20:57:38.514 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_signature PASSED [ 44%] 2024-05-09T20:57:39.766 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_access_key PASSED [ 45%] 2024-05-09T20:57:41.006 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_date_format PASSED [ 45%] 2024-05-09T20:57:42.282 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_no_key_specified PASSED [ 45%] 2024-05-09T20:57:43.514 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_signature PASSED [ 45%] 2024-05-09T20:57:44.842 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_policy_condition PASSED [ 45%] 2024-05-09T20:57:46.354 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_user_specified_header PASSED [ 45%] 2024-05-09T20:57:47.634 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_request_missing_policy_specified_field PASSED [ 45%] 2024-05-09T20:57:48.890 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_condition_is_case_sensitive PASSED [ 46%] 2024-05-09T20:57:50.126 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expires_is_case_sensitive PASSED [ 46%] 2024-05-09T20:57:51.411 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_expired_policy PASSED [ 46%] 2024-05-09T20:57:53.274 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_wrong_bucket PASSED [ 46%] 2024-05-09T20:57:54.554 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_request_field_value PASSED [ 46%] 2024-05-09T20:57:55.790 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_expires_condition PASSED [ 46%] 2024-05-09T20:57:57.106 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_conditions_list PASSED [ 46%] 2024-05-09T20:57:58.363 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_limit_exceeded PASSED [ 46%] 2024-05-09T20:57:59.634 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_missing_content_length_argument PASSED [ 47%] 2024-05-09T20:58:00.902 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_invalid_content_length_argument PASSED [ 47%] 2024-05-09T20:58:02.207 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_below_minimum PASSED [ 47%] 2024-05-09T20:58:04.327 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_upload_size_rgw_chunk_size_bug PASSED [ 47%] 2024-05-09T20:58:05.567 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_post_object_empty_conditions PASSED [ 47%] 2024-05-09T20:58:07.110 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_good PASSED [ 47%] 2024-05-09T20:58:08.630 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmatch_failed PASSED [ 47%] 2024-05-09T20:58:10.162 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_good PASSED [ 48%] 2024-05-09T20:58:11.714 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifnonematch_failed PASSED [ 48%] 2024-05-09T20:58:13.214 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_good PASSED [ 48%] 2024-05-09T20:58:15.731 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifmodifiedsince_failed PASSED [ 48%] 2024-05-09T20:58:17.282 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_good PASSED [ 48%] 2024-05-09T20:58:18.794 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_get_object_ifunmodifiedsince_failed PASSED [ 48%] 2024-05-09T20:58:20.478 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_good PASSED [ 48%] 2024-05-09T20:58:22.238 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_failed PASSED [ 49%] 2024-05-09T20:58:23.934 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_overwrite_existed_good PASSED [ 49%] 2024-05-09T20:58:25.370 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifmatch_nonexisted_failed PASSED [ 49%] 2024-05-09T20:58:27.110 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_good PASSED [ 49%] 2024-05-09T20:58:28.850 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_failed PASSED [ 49%] 2024-05-09T20:58:30.358 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_nonexisted_good PASSED [ 49%] 2024-05-09T20:58:32.127 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_put_object_ifnonmatch_overwrite_existed_failed PASSED [ 49%] 2024-05-09T20:58:33.550 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get PASSED [ 50%] 2024-05-09T20:58:34.850 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_gone PASSED [ 50%] 2024-05-09T20:58:36.135 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_delete_key_bucket_gone PASSED [ 50%] 2024-05-09T20:58:37.838 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_gone PASSED [ 50%] 2024-05-09T20:58:39.118 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head PASSED [ 50%] 2024-05-09T20:58:39.726 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_notexist PASSED [ 50%] 2024-05-09T20:58:41.766 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_head_extended PASSED [ 50%] 2024-05-09T20:58:43.186 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_bucket_acl PASSED [ 50%] 2024-05-09T20:58:44.634 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_object_acl PASSED [ 51%] 2024-05-09T20:58:49.086 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_acl_mtime PASSED [ 51%] 2024-05-09T20:58:50.559 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated PASSED [ 51%] 2024-05-09T20:58:52.310 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_response_headers PASSED [ 51%] 2024-05-09T20:58:53.774 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_acl PASSED [ 51%] 2024-05-09T20:58:55.246 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_acl PASSED [ 51%] 2024-05-09T20:58:56.579 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_bucket_gone PASSED [ 51%] 2024-05-09T20:58:58.086 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_authenticated_object_gone PASSED [ 52%] 2024-05-09T20:58:59.631 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired PASSED [ 52%] 2024-05-09T20:59:01.198 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_not_expired_tenant PASSED [ 52%] 2024-05-09T20:59:02.675 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_range_zero PASSED [ 52%] 2024-05-09T20:59:04.086 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_max_range PASSED [ 52%] 2024-05-09T20:59:05.486 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_get_x_amz_expires_out_positive_range PASSED [ 52%] 2024-05-09T20:59:07.067 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put PASSED [ 52%] 2024-05-09T20:59:08.519 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_anon_put_write_access PASSED [ 53%] 2024-05-09T20:59:09.922 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_put_authenticated PASSED [ 53%] 2024-05-09T20:59:11.370 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl PASSED [ 53%] 2024-05-09T20:59:12.907 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_presigned_put_object_with_acl_tenant PASSED [ 53%] 2024-05-09T20:59:14.338 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_raw_put_authenticated_expired PASSED [ 53%] 2024-05-09T20:59:14.950 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_starts_nonalpha PASSED [ 53%] 2024-05-09T20:59:15.562 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_one PASSED [ 53%] 2024-05-09T20:59:16.183 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_short_two PASSED [ 53%] 2024-05-09T20:59:17.415 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_60 PASSED [ 54%] 2024-05-09T20:59:18.614 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_61 PASSED [ 54%] 2024-05-09T20:59:19.822 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_62 PASSED [ 54%] 2024-05-09T20:59:21.006 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_long_63 PASSED [ 54%] 2024-05-09T20:59:22.406 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_long_name PASSED [ 54%] 2024-05-09T20:59:23.022 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_bad_ip PASSED [ 54%] 2024-05-09T20:59:23.678 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_underscore PASSED [ 54%] 2024-05-09T20:59:24.874 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_long PASSED [ 55%] 2024-05-09T20:59:25.486 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_at_end PASSED [ 55%] 2024-05-09T20:59:26.105 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dot PASSED [ 55%] 2024-05-09T20:59:26.718 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dot_dash PASSED [ 55%] 2024-05-09T20:59:27.399 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_dns_dash_dot PASSED [ 55%] 2024-05-09T20:59:28.750 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists PASSED [ 55%] 2024-05-09T20:59:30.046 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_get_location PASSED [ 55%] 2024-05-09T20:59:31.315 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_exists_nonowner PASSED [ 56%] 2024-05-09T20:59:32.735 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_overwrite_acl PASSED [ 56%] 2024-05-09T20:59:34.019 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_new_acl PASSED [ 56%] 2024-05-09T20:59:35.270 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_default PASSED [ 56%] 2024-05-09T20:59:36.535 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_during_create PASSED [ 56%] 2024-05-09T20:59:38.027 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned PASSED [ 56%] 2024-05-09T20:59:39.334 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_publicreadwrite PASSED [ 56%] 2024-05-09T20:59:40.615 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_authenticatedread PASSED [ 56%] 2024-05-09T20:59:42.142 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_default PASSED [ 57%] 2024-05-09T20:59:43.690 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_during_create PASSED [ 57%] 2024-05-09T20:59:45.375 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned PASSED [ 57%] 2024-05-09T20:59:46.931 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_publicreadwrite PASSED [ 57%] 2024-05-09T20:59:48.459 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_authenticatedread PASSED [ 57%] 2024-05-09T20:59:50.146 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerread PASSED [ 57%] 2024-05-09T20:59:51.783 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_canned_bucketownerfullcontrol PASSED [ 57%] 2024-05-09T20:59:53.575 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_owner PASSED [ 58%] 2024-05-09T20:59:55.302 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_full_control_verify_attributes PASSED [ 58%] 2024-05-09T20:59:56.575 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_canned_private_to_private PASSED [ 58%] 2024-05-09T20:59:58.331 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl PASSED [ 58%] 2024-05-09T21:00:00.083 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_write PASSED [ 58%] 2024-05-09T21:00:01.751 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_writeacp PASSED [ 58%] 2024-05-09T21:00:03.459 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_read PASSED [ 58%] 2024-05-09T21:00:05.182 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_acl_readacp PASSED [ 59%] 2024-05-09T21:00:07.151 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_fullcontrol PASSED [ 59%] 2024-05-09T21:00:08.927 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_read PASSED [ 59%] 2024-05-09T21:00:10.718 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_readacp PASSED [ 59%] 2024-05-09T21:00:12.602 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_write PASSED [ 59%] 2024-05-09T21:00:14.371 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_userid_writeacp PASSED [ 59%] 2024-05-09T21:00:15.695 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_nonexist_user PASSED [ 59%] 2024-05-09T21:00:17.223 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_header_acl_grants PASSED [ 59%] 2024-05-09T21:00:18.855 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_header_acl_grants PASSED [ 60%] 2024-05-09T21:00:20.339 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email PASSED [ 60%] 2024-05-09T21:00:21.711 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_grant_email_not_exist PASSED [ 60%] 2024-05-09T21:00:23.518 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_acl_revoke_all PASSED [ 60%] 2024-05-09T21:00:25.735 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_private PASSED [ 60%] 2024-05-09T21:00:28.035 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_private PASSED [ 60%] 2024-05-09T21:00:30.271 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicread PASSED [ 60%] 2024-05-09T21:00:32.538 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicread PASSED [ 61%] 2024-05-09T21:00:35.099 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_object_publicreadwrite PASSED [ 61%] 2024-05-09T21:00:37.703 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_private_objectv2_publicreadwrite PASSED [ 61%] 2024-05-09T21:00:40.099 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_private PASSED [ 61%] 2024-05-09T21:00:42.335 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicread PASSED [ 61%] 2024-05-09T21:00:44.639 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicread_object_publicreadwrite PASSED [ 61%] 2024-05-09T21:00:47.026 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_private PASSED [ 61%] 2024-05-09T21:00:49.426 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicread PASSED [ 62%] 2024-05-09T21:00:51.771 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_access_bucket_publicreadwrite_object_publicreadwrite PASSED [ 62%] 2024-05-09T21:00:55.811 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_create_then_list PASSED [ 62%] 2024-05-09T21:00:59.651 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_buckets_list_ctime PASSED [ 62%] 2024-05-09T21:01:00.231 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_anonymous PASSED [ 62%] 2024-05-09T21:01:00.843 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_invalid_auth PASSED [ 62%] 2024-05-09T21:01:01.439 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_list_buckets_bad_auth PASSED [ 62%] 2024-05-09T21:01:02.763 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_alpha PASSED [ 62%] 2024-05-09T21:01:04.123 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_starts_digit PASSED [ 63%] 2024-05-09T21:01:05.323 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_period PASSED [ 63%] 2024-05-09T21:01:06.515 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_naming_good_contains_hyphen PASSED [ 63%] 2024-05-09T21:01:08.507 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_recreate_not_overriding PASSED [ 63%] 2024-05-09T21:01:13.735 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_create_special_key_names PASSED [ 63%] 2024-05-09T21:01:15.827 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_bucket_list_special_prefix PASSED [ 63%] 2024-05-09T21:01:17.611 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_zero_size PASSED [ 63%] 2024-05-09T21:01:22.111 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_16m PASSED [ 64%] 2024-05-09T21:01:23.903 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_same_bucket PASSED [ 64%] 2024-05-09T21:01:25.639 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_verify_contenttype PASSED [ 64%] 2024-05-09T21:01:27.235 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself PASSED [ 64%] 2024-05-09T21:01:29.127 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_to_itself_with_metadata PASSED [ 64%] 2024-05-09T21:01:31.575 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_diff_bucket PASSED [ 64%] 2024-05-09T21:01:33.647 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_bucket PASSED [ 64%] 2024-05-09T21:01:35.771 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_not_owned_object_bucket PASSED [ 65%] 2024-05-09T21:01:37.671 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_canned_acl PASSED [ 65%] 2024-05-09T21:01:41.053 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_retaining_metadata PASSED [ 65%] 2024-05-09T21:01:44.239 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_replacing_metadata PASSED [ 65%] 2024-05-09T21:01:45.479 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_bucket_not_found PASSED [ 65%] 2024-05-09T21:01:46.735 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_key_not_found PASSED [ 65%] 2024-05-09T21:01:51.851 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_bucket PASSED [ 65%] 2024-05-09T21:01:53.931 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioned_url_encoding PASSED [ 65%] 2024-05-09T21:02:11.095 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_object_copy_versioning_multipart_upload PASSED [ 66%] 2024-05-09T21:02:12.547 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_empty PASSED [ 66%] 2024-05-09T21:02:14.391 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_small PASSED [ 66%] 2024-05-09T21:02:19.400 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_small PASSED [ 66%] 2024-05-09T21:02:21.119 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_invalid_range PASSED [ 66%] 2024-05-09T21:02:24.123 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_without_range PASSED [ 66%] 2024-05-09T21:02:36.359 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_special_names PASSED [ 66%] 2024-05-09T21:02:53.191 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload PASSED [ 67%] 2024-05-09T21:03:16.336 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_versioned PASSED [ 67%] 2024-05-09T21:04:49.036 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_resend_part PASSED [ 67%] 2024-05-09T21:05:01.039 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_multiple_sizes PASSED [ 67%] 2024-05-09T21:05:21.008 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_copy_multiple_sizes PASSED [ 67%] 2024-05-09T21:05:23.628 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_size_too_small PASSED [ 67%] 2024-05-09T21:05:28.900 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_contents PASSED [ 67%] 2024-05-09T21:05:34.724 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_overwrite_existing_object PASSED [ 68%] 2024-05-09T21:05:38.116 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload PASSED [ 68%] 2024-05-09T21:05:39.596 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_abort_multipart_upload_not_found PASSED [ 68%] 2024-05-09T21:05:45.120 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload PASSED [ 68%] 2024-05-09T21:05:46.971 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_list_multipart_upload_owner PASSED [ 68%] 2024-05-09T21:05:48.520 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_missing_part PASSED [ 68%] 2024-05-09T21:05:50.072 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_upload_incorrect_etag PASSED [ 68%] 2024-05-09T21:06:10.328 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_multipart_get_part PASSED [ 68%] 2024-05-09T21:06:11.944 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_non_multipart_get_part PASSED [ 69%] 2024-05-09T21:06:13.332 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_100_continue PASSED [ 69%] 2024-05-09T21:06:15.016 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_set_cors PASSED [ 69%] 2024-05-09T21:06:16.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T21:06:16.420 INFO:teuthology.orchestra.run.smithi063.stdout:s3tests_boto3/functional/test_s3.py::test_cors_origin_response 2024-05-09T21:06:16.421 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi063 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-05-09T21:06:16.422 INFO:tasks.s3tests:Cleaning up boto... 2024-05-09T21:06:16.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T21:06:16.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T21:06:16.933 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-05-09T21:06:17.117 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-05-09T21:06:17.118 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-05-09T21:06:17.119 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-05-09T21:06:17.349 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-05-09T21:06:17.350 DEBUG:teuthology.orchestra.run.smithi063:> rm /home/ubuntu/cephtest/boto-client.0.cfg 2024-05-09T21:06:17.395 INFO:tasks.s3tests:Removing s3-tests... 2024-05-09T21:06:17.395 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/s3-tests-client.0 2024-05-09T21:06:17.570 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi063 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-05-09T21:06:20.390 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=205c004cebd1426ca898990fec779345 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 736, in task with contextutil.nested( File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 61, in download yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/contextutil.py", line 30, in nested vars.append(enter()) File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 462, in run_tests toxvenv_sh(ctx, remote, args, label="s3 tests against rgw") File "/home/teuthworker/src/github.com_samarahu_ceph_86872c6b9201173c508ab909aaa65bdb12239dd6/qa/tasks/s3tests.py", line 405, in toxvenv_sh return remote.sh(['source', activate, run.Raw('&&')] + args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 174, in _raise_for_status raise ConnectionLostError(command=self.command, teuthology.exceptions.ConnectionLostError: SSH connection to smithi063 was lost: "source /home/ubuntu/cephtest/tox-venv/bin/activate && cd /home/ubuntu/cephtest/s3-tests-client.0 && S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto-client.0.cfg REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt tox -- -v -m 'not fails_on_rgw and not lifecycle_expiration and not lifecycle_transition and not cloud_transition and not test_of_sts and not webidentity_test and not fails_with_subdomain and not sse_s3'" 2024-05-09T21:06:20.393 DEBUG:teuthology.run_tasks:Unwinding manager s3tests 2024-05-09T21:06:20.405 DEBUG:teuthology.run_tasks:Unwinding manager s3tests-java 2024-05-09T21:06:20.413 DEBUG:tasks.s3tests_java:S3 Tests Java: END 2024-05-09T21:06:20.413 INFO:tasks.s3tests_java:S3 Tests Java: Cleaning up s3-tests-java... 2024-05-09T21:06:20.414 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/s3-tests-java 2024-05-09T21:06:20.468 INFO:tasks.s3tests_java:S3 Tests Java: Deleting S3 users... 2024-05-09T21:06:20.468 DEBUG:teuthology.orchestra.run.smithi063:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid tester.client.0 --purge-data --cluster ceph 2024-05-09T21:06:20.548 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setuser ceph since I am not root 2024-05-09T21:06:20.549 INFO:teuthology.orchestra.run.smithi063.stderr:ignoring --setgroup ceph since I am not root 2024-05-09T21:06:22.119 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2024-05-09T21:06:22.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2024-05-09T21:06:22.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2024-05-09T21:06:22.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~5s 2024-05-09T21:06:22.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~5s 2024-05-09T21:06:22.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~5s 2024-05-09T21:06:22.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~5s 2024-05-09T21:06:22.120 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~5s 2024-05-09T21:06:22.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~5s 2024-05-09T21:06:22.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~5s 2024-05-09T21:06:22.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~5s 2024-05-09T21:06:22.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~5s 2024-05-09T21:06:22.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~5s 2024-05-09T21:06:22.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~5s 2024-05-09T21:06:27.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~10s 2024-05-09T21:06:27.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~10s 2024-05-09T21:06:27.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~10s 2024-05-09T21:06:27.123 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~10s 2024-05-09T21:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~10s 2024-05-09T21:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~10s 2024-05-09T21:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~10s 2024-05-09T21:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~10s 2024-05-09T21:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~10s 2024-05-09T21:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~10s 2024-05-09T21:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~10s 2024-05-09T21:06:27.124 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~10s 2024-05-09T21:06:27.125 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~10s 2024-05-09T21:06:27.125 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~10s 2024-05-09T21:06:32.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~15s 2024-05-09T21:06:32.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~15s 2024-05-09T21:06:32.127 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~15s 2024-05-09T21:06:32.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~15s 2024-05-09T21:06:32.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~15s 2024-05-09T21:06:32.128 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~15s 2024-05-09T21:06:37.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~20s 2024-05-09T21:06:37.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~20s 2024-05-09T21:06:37.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~20s 2024-05-09T21:06:37.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~20s 2024-05-09T21:06:37.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~20s 2024-05-09T21:06:37.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~20s 2024-05-09T21:06:37.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~20s 2024-05-09T21:06:37.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~20s 2024-05-09T21:06:37.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~20s 2024-05-09T21:06:37.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~20s 2024-05-09T21:06:37.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~20s 2024-05-09T21:06:37.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~20s 2024-05-09T21:06:37.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~20s 2024-05-09T21:06:37.131 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~20s 2024-05-09T21:06:42.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~25s 2024-05-09T21:06:42.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~25s 2024-05-09T21:06:42.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~25s 2024-05-09T21:06:42.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~25s 2024-05-09T21:06:42.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~25s 2024-05-09T21:06:42.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~25s 2024-05-09T21:06:42.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~25s 2024-05-09T21:06:42.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~25s 2024-05-09T21:06:42.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~25s 2024-05-09T21:06:42.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~25s 2024-05-09T21:06:42.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~25s 2024-05-09T21:06:42.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~25s 2024-05-09T21:06:42.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~25s 2024-05-09T21:06:42.135 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~25s 2024-05-09T21:06:47.136 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~30s 2024-05-09T21:06:47.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~30s 2024-05-09T21:06:47.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~30s 2024-05-09T21:06:47.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~30s 2024-05-09T21:06:47.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~30s 2024-05-09T21:06:47.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~30s 2024-05-09T21:06:47.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~30s 2024-05-09T21:06:47.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~30s 2024-05-09T21:06:47.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~30s 2024-05-09T21:06:47.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~30s 2024-05-09T21:06:47.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~30s 2024-05-09T21:06:47.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~30s 2024-05-09T21:06:47.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~30s 2024-05-09T21:06:47.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~30s 2024-05-09T21:06:52.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~35s 2024-05-09T21:06:52.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~35s 2024-05-09T21:06:52.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~35s 2024-05-09T21:06:52.140 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~35s 2024-05-09T21:06:52.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~35s 2024-05-09T21:06:52.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~35s 2024-05-09T21:06:52.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~35s 2024-05-09T21:06:52.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~35s 2024-05-09T21:06:52.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~35s 2024-05-09T21:06:52.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~35s 2024-05-09T21:06:52.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~35s 2024-05-09T21:06:52.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~35s 2024-05-09T21:06:52.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~35s 2024-05-09T21:06:52.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~35s 2024-05-09T21:06:57.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~40s 2024-05-09T21:06:57.143 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~40s 2024-05-09T21:06:57.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~40s 2024-05-09T21:06:57.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~40s 2024-05-09T21:06:57.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~40s 2024-05-09T21:06:57.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~40s 2024-05-09T21:06:57.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~40s 2024-05-09T21:06:57.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~40s 2024-05-09T21:06:57.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~40s 2024-05-09T21:06:57.144 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~40s 2024-05-09T21:06:57.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~40s 2024-05-09T21:06:57.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~40s 2024-05-09T21:06:57.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~40s 2024-05-09T21:06:57.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~40s 2024-05-09T21:07:02.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~45s 2024-05-09T21:07:02.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~45s 2024-05-09T21:07:02.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~45s 2024-05-09T21:07:02.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~45s 2024-05-09T21:07:02.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~45s 2024-05-09T21:07:02.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~45s 2024-05-09T21:07:07.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~50s 2024-05-09T21:07:07.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~50s 2024-05-09T21:07:07.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~50s 2024-05-09T21:07:07.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~50s 2024-05-09T21:07:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~50s 2024-05-09T21:07:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~50s 2024-05-09T21:07:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~50s 2024-05-09T21:07:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~50s 2024-05-09T21:07:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~50s 2024-05-09T21:07:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~50s 2024-05-09T21:07:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~50s 2024-05-09T21:07:07.150 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~50s 2024-05-09T21:07:07.151 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~50s 2024-05-09T21:07:07.151 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~50s 2024-05-09T21:07:12.152 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~55s 2024-05-09T21:07:12.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~55s 2024-05-09T21:07:12.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~55s 2024-05-09T21:07:12.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~55s 2024-05-09T21:07:12.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~55s 2024-05-09T21:07:12.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~55s 2024-05-09T21:07:12.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~55s 2024-05-09T21:07:12.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~55s 2024-05-09T21:07:12.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~55s 2024-05-09T21:07:12.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~55s 2024-05-09T21:07:12.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~55s 2024-05-09T21:07:12.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~55s 2024-05-09T21:07:12.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~55s 2024-05-09T21:07:12.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~55s 2024-05-09T21:07:17.156 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~60s 2024-05-09T21:07:17.156 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~60s 2024-05-09T21:07:17.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~60s 2024-05-09T21:07:17.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~60s 2024-05-09T21:07:17.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~60s 2024-05-09T21:07:17.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~60s 2024-05-09T21:07:17.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~60s 2024-05-09T21:07:17.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~60s 2024-05-09T21:07:17.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~60s 2024-05-09T21:07:17.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~60s 2024-05-09T21:07:17.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~60s 2024-05-09T21:07:17.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~60s 2024-05-09T21:07:17.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~60s 2024-05-09T21:07:17.158 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~60s 2024-05-09T21:07:22.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~65s 2024-05-09T21:07:22.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~65s 2024-05-09T21:07:22.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2024-05-09T21:07:22.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~65s 2024-05-09T21:07:22.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~65s 2024-05-09T21:07:22.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~65s 2024-05-09T21:07:22.160 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~65s 2024-05-09T21:07:22.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~65s 2024-05-09T21:07:22.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~65s 2024-05-09T21:07:22.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~65s 2024-05-09T21:07:22.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~65s 2024-05-09T21:07:22.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~65s 2024-05-09T21:07:22.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~65s 2024-05-09T21:07:22.161 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~65s 2024-05-09T21:07:27.162 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~70s 2024-05-09T21:07:27.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~70s 2024-05-09T21:07:27.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~70s 2024-05-09T21:07:27.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~70s 2024-05-09T21:07:27.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~70s 2024-05-09T21:07:27.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~70s 2024-05-09T21:07:27.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~70s 2024-05-09T21:07:27.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~70s 2024-05-09T21:07:27.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~70s 2024-05-09T21:07:27.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~70s 2024-05-09T21:07:27.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~70s 2024-05-09T21:07:27.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~70s 2024-05-09T21:07:27.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~70s 2024-05-09T21:07:27.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~70s 2024-05-09T21:07:32.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~75s 2024-05-09T21:07:32.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~75s 2024-05-09T21:07:32.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~75s 2024-05-09T21:07:32.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~75s 2024-05-09T21:07:32.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~75s 2024-05-09T21:07:32.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~75s 2024-05-09T21:07:32.166 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~75s 2024-05-09T21:07:32.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~75s 2024-05-09T21:07:32.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~75s 2024-05-09T21:07:32.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~75s 2024-05-09T21:07:32.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~75s 2024-05-09T21:07:32.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~75s 2024-05-09T21:07:32.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~75s 2024-05-09T21:07:32.167 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~75s 2024-05-09T21:07:37.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~80s 2024-05-09T21:07:37.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~80s 2024-05-09T21:07:37.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~80s 2024-05-09T21:07:37.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~80s 2024-05-09T21:07:37.169 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~80s 2024-05-09T21:07:37.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~80s 2024-05-09T21:07:37.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~80s 2024-05-09T21:07:37.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~80s 2024-05-09T21:07:37.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~80s 2024-05-09T21:07:37.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~80s 2024-05-09T21:07:37.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~80s 2024-05-09T21:07:37.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~80s 2024-05-09T21:07:37.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~80s 2024-05-09T21:07:37.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~80s 2024-05-09T21:07:42.172 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~85s 2024-05-09T21:07:42.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~85s 2024-05-09T21:07:42.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~85s 2024-05-09T21:07:42.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~85s 2024-05-09T21:07:42.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~85s 2024-05-09T21:07:42.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~85s 2024-05-09T21:07:42.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~85s 2024-05-09T21:07:42.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~85s 2024-05-09T21:07:42.173 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~85s 2024-05-09T21:07:42.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~85s 2024-05-09T21:07:42.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~85s 2024-05-09T21:07:42.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~85s 2024-05-09T21:07:42.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~85s 2024-05-09T21:07:42.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~85s 2024-05-09T21:07:47.175 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~90s 2024-05-09T21:07:47.175 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~90s 2024-05-09T21:07:47.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~90s 2024-05-09T21:07:47.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~90s 2024-05-09T21:07:47.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~90s 2024-05-09T21:07:47.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~90s 2024-05-09T21:07:47.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~90s 2024-05-09T21:07:47.176 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~90s 2024-05-09T21:07:47.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~90s 2024-05-09T21:07:47.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~90s 2024-05-09T21:07:47.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~90s 2024-05-09T21:07:47.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~90s 2024-05-09T21:07:47.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~90s 2024-05-09T21:07:47.177 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~90s 2024-05-09T21:07:52.178 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~95s 2024-05-09T21:07:52.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~95s 2024-05-09T21:07:52.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~95s 2024-05-09T21:07:52.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~95s 2024-05-09T21:07:52.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~95s 2024-05-09T21:07:52.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~95s 2024-05-09T21:07:52.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~95s 2024-05-09T21:07:52.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~95s 2024-05-09T21:07:52.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~95s 2024-05-09T21:07:52.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~95s 2024-05-09T21:07:52.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~95s 2024-05-09T21:07:52.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~95s 2024-05-09T21:07:52.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~95s 2024-05-09T21:07:52.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~95s 2024-05-09T21:07:57.181 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~100s 2024-05-09T21:07:57.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~100s 2024-05-09T21:07:57.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~100s 2024-05-09T21:07:57.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~100s 2024-05-09T21:07:57.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~100s 2024-05-09T21:07:57.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~100s 2024-05-09T21:07:57.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~100s 2024-05-09T21:07:57.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~100s 2024-05-09T21:07:57.182 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~100s 2024-05-09T21:07:57.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~100s 2024-05-09T21:07:57.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~100s 2024-05-09T21:07:57.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~100s 2024-05-09T21:07:57.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~100s 2024-05-09T21:07:57.183 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~100s 2024-05-09T21:08:02.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~105s 2024-05-09T21:08:02.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~105s 2024-05-09T21:08:02.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~105s 2024-05-09T21:08:02.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~105s 2024-05-09T21:08:02.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~105s 2024-05-09T21:08:02.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~105s 2024-05-09T21:08:02.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~105s 2024-05-09T21:08:02.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~105s 2024-05-09T21:08:02.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~105s 2024-05-09T21:08:02.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~105s 2024-05-09T21:08:02.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~105s 2024-05-09T21:08:02.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~105s 2024-05-09T21:08:02.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~105s 2024-05-09T21:08:02.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~105s 2024-05-09T21:08:07.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~110s 2024-05-09T21:08:07.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~110s 2024-05-09T21:08:07.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~110s 2024-05-09T21:08:07.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~110s 2024-05-09T21:08:07.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~110s 2024-05-09T21:08:07.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~110s 2024-05-09T21:08:07.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~110s 2024-05-09T21:08:07.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~110s 2024-05-09T21:08:07.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~110s 2024-05-09T21:08:07.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~110s 2024-05-09T21:08:07.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~110s 2024-05-09T21:08:07.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~110s 2024-05-09T21:08:07.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~110s 2024-05-09T21:08:07.189 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~110s 2024-05-09T21:08:12.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~115s 2024-05-09T21:08:12.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~115s 2024-05-09T21:08:12.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~115s 2024-05-09T21:08:12.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~115s 2024-05-09T21:08:12.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~115s 2024-05-09T21:08:12.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~115s 2024-05-09T21:08:12.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~115s 2024-05-09T21:08:12.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~115s 2024-05-09T21:08:12.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~115s 2024-05-09T21:08:12.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~115s 2024-05-09T21:08:12.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~115s 2024-05-09T21:08:12.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~115s 2024-05-09T21:08:12.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~115s 2024-05-09T21:08:12.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~115s 2024-05-09T21:08:17.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~120s 2024-05-09T21:08:17.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~120s 2024-05-09T21:08:17.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~120s 2024-05-09T21:08:17.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~120s 2024-05-09T21:08:17.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~120s 2024-05-09T21:08:17.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~120s 2024-05-09T21:08:17.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~120s 2024-05-09T21:08:17.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~120s 2024-05-09T21:08:17.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~120s 2024-05-09T21:08:17.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~120s 2024-05-09T21:08:17.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~120s 2024-05-09T21:08:17.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~120s 2024-05-09T21:08:17.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~120s 2024-05-09T21:08:17.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~120s 2024-05-09T21:08:22.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~125s 2024-05-09T21:08:22.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~125s 2024-05-09T21:08:22.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~125s 2024-05-09T21:08:22.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~125s 2024-05-09T21:08:22.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~125s 2024-05-09T21:08:22.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~125s 2024-05-09T21:08:27.199 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~130s 2024-05-09T21:08:27.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~130s 2024-05-09T21:08:27.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~130s 2024-05-09T21:08:27.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~130s 2024-05-09T21:08:27.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~130s 2024-05-09T21:08:27.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~130s 2024-05-09T21:08:27.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~130s 2024-05-09T21:08:27.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~130s 2024-05-09T21:08:27.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~130s 2024-05-09T21:08:27.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~130s 2024-05-09T21:08:27.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~130s 2024-05-09T21:08:27.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~130s 2024-05-09T21:08:27.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~130s 2024-05-09T21:08:27.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~130s 2024-05-09T21:08:32.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~135s 2024-05-09T21:08:32.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~135s 2024-05-09T21:08:32.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~135s 2024-05-09T21:08:32.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~135s 2024-05-09T21:08:32.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~135s 2024-05-09T21:08:32.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~135s 2024-05-09T21:08:32.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~135s 2024-05-09T21:08:32.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~135s 2024-05-09T21:08:32.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~135s 2024-05-09T21:08:32.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~135s 2024-05-09T21:08:32.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~135s 2024-05-09T21:08:32.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~135s 2024-05-09T21:08:32.204 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~135s 2024-05-09T21:08:32.205 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~135s 2024-05-09T21:08:37.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~140s 2024-05-09T21:08:37.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~140s 2024-05-09T21:08:37.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~140s 2024-05-09T21:08:37.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~140s 2024-05-09T21:08:37.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~140s 2024-05-09T21:08:37.208 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~140s 2024-05-09T21:08:42.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~145s 2024-05-09T21:08:42.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~145s 2024-05-09T21:08:42.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~145s 2024-05-09T21:08:42.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~145s 2024-05-09T21:08:42.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~145s 2024-05-09T21:08:42.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~145s 2024-05-09T21:08:42.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~145s 2024-05-09T21:08:42.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~145s 2024-05-09T21:08:42.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~145s 2024-05-09T21:08:42.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~145s 2024-05-09T21:08:42.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~145s 2024-05-09T21:08:42.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~145s 2024-05-09T21:08:42.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~145s 2024-05-09T21:08:42.211 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~145s 2024-05-09T21:08:47.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~150s 2024-05-09T21:08:47.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~150s 2024-05-09T21:08:47.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~150s 2024-05-09T21:08:47.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~150s 2024-05-09T21:08:47.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~150s 2024-05-09T21:08:47.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~150s 2024-05-09T21:08:47.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~150s 2024-05-09T21:08:47.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~150s 2024-05-09T21:08:47.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~150s 2024-05-09T21:08:47.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~150s 2024-05-09T21:08:47.214 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~150s 2024-05-09T21:08:47.215 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~150s 2024-05-09T21:08:47.215 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~150s 2024-05-09T21:08:47.215 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~150s 2024-05-09T21:08:52.216 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~155s 2024-05-09T21:08:52.216 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~155s 2024-05-09T21:08:52.216 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~155s 2024-05-09T21:08:52.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~155s 2024-05-09T21:08:52.218 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~155s 2024-05-09T21:08:57.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~160s 2024-05-09T21:08:57.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~160s 2024-05-09T21:08:57.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~160s 2024-05-09T21:08:57.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~160s 2024-05-09T21:08:57.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~160s 2024-05-09T21:08:57.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~160s 2024-05-09T21:08:57.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~160s 2024-05-09T21:08:57.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~160s 2024-05-09T21:08:57.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~160s 2024-05-09T21:08:57.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~160s 2024-05-09T21:08:57.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~160s 2024-05-09T21:08:57.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~160s 2024-05-09T21:08:57.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~160s 2024-05-09T21:08:57.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~160s 2024-05-09T21:09:02.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~165s 2024-05-09T21:09:02.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~165s 2024-05-09T21:09:02.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~165s 2024-05-09T21:09:02.223 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~165s 2024-05-09T21:09:02.224 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~165s 2024-05-09T21:09:02.224 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~165s 2024-05-09T21:09:07.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~170s 2024-05-09T21:09:07.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~170s 2024-05-09T21:09:07.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~170s 2024-05-09T21:09:07.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~170s 2024-05-09T21:09:07.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~170s 2024-05-09T21:09:07.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~170s 2024-05-09T21:09:07.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~170s 2024-05-09T21:09:07.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~170s 2024-05-09T21:09:07.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~170s 2024-05-09T21:09:07.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~170s 2024-05-09T21:09:07.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~170s 2024-05-09T21:09:07.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~170s 2024-05-09T21:09:07.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~170s 2024-05-09T21:09:07.227 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~170s 2024-05-09T21:09:12.228 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~175s 2024-05-09T21:09:12.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~175s 2024-05-09T21:09:12.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~175s 2024-05-09T21:09:12.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~175s 2024-05-09T21:09:12.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~175s 2024-05-09T21:09:12.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~175s 2024-05-09T21:09:12.229 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~175s 2024-05-09T21:09:12.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~175s 2024-05-09T21:09:12.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~175s 2024-05-09T21:09:12.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~175s 2024-05-09T21:09:12.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~175s 2024-05-09T21:09:12.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~175s 2024-05-09T21:09:12.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~175s 2024-05-09T21:09:12.230 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~175s 2024-05-09T21:09:17.232 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~180s 2024-05-09T21:09:17.232 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~180s 2024-05-09T21:09:17.232 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~180s 2024-05-09T21:09:17.232 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~180s 2024-05-09T21:09:17.232 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~180s 2024-05-09T21:09:17.233 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~180s 2024-05-09T21:09:22.234 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~185s 2024-05-09T21:09:22.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~185s 2024-05-09T21:09:22.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~185s 2024-05-09T21:09:22.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~185s 2024-05-09T21:09:22.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~185s 2024-05-09T21:09:22.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~185s 2024-05-09T21:09:22.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~185s 2024-05-09T21:09:22.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~185s 2024-05-09T21:09:22.235 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~185s 2024-05-09T21:09:22.236 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~185s 2024-05-09T21:09:22.236 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~185s 2024-05-09T21:09:22.236 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~185s 2024-05-09T21:09:22.236 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~185s 2024-05-09T21:09:22.236 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~185s 2024-05-09T21:09:27.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~190s 2024-05-09T21:09:27.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~190s 2024-05-09T21:09:27.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~190s 2024-05-09T21:09:27.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~190s 2024-05-09T21:09:27.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~190s 2024-05-09T21:09:27.238 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~190s 2024-05-09T21:09:27.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~190s 2024-05-09T21:09:27.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~190s 2024-05-09T21:09:27.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~190s 2024-05-09T21:09:27.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~190s 2024-05-09T21:09:27.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~190s 2024-05-09T21:09:27.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~190s 2024-05-09T21:09:27.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~190s 2024-05-09T21:09:27.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~190s 2024-05-09T21:09:32.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~195s 2024-05-09T21:09:32.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~195s 2024-05-09T21:09:32.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~195s 2024-05-09T21:09:32.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~195s 2024-05-09T21:09:32.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~195s 2024-05-09T21:09:32.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~195s 2024-05-09T21:09:32.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~195s 2024-05-09T21:09:32.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~195s 2024-05-09T21:09:32.242 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~195s 2024-05-09T21:09:32.243 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~195s 2024-05-09T21:09:32.243 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~195s 2024-05-09T21:09:32.243 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~195s 2024-05-09T21:09:32.243 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~195s 2024-05-09T21:09:32.243 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~195s 2024-05-09T21:09:37.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~200s 2024-05-09T21:09:37.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~200s 2024-05-09T21:09:37.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~200s 2024-05-09T21:09:37.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~200s 2024-05-09T21:09:37.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~200s 2024-05-09T21:09:37.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~200s 2024-05-09T21:09:37.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~200s 2024-05-09T21:09:37.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~200s 2024-05-09T21:09:37.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~200s 2024-05-09T21:09:37.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~200s 2024-05-09T21:09:37.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~200s 2024-05-09T21:09:37.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~200s 2024-05-09T21:09:37.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~200s 2024-05-09T21:09:37.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~200s 2024-05-09T21:09:42.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~205s 2024-05-09T21:09:42.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~205s 2024-05-09T21:09:42.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~205s 2024-05-09T21:09:42.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~205s 2024-05-09T21:09:42.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~205s 2024-05-09T21:09:42.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~205s 2024-05-09T21:09:42.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~205s 2024-05-09T21:09:42.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~205s 2024-05-09T21:09:42.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~205s 2024-05-09T21:09:42.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~205s 2024-05-09T21:09:42.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~205s 2024-05-09T21:09:42.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~205s 2024-05-09T21:09:42.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~205s 2024-05-09T21:09:42.249 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~205s 2024-05-09T21:09:47.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~210s 2024-05-09T21:09:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~210s 2024-05-09T21:09:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~210s 2024-05-09T21:09:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~210s 2024-05-09T21:09:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~210s 2024-05-09T21:09:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~210s 2024-05-09T21:09:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~210s 2024-05-09T21:09:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~210s 2024-05-09T21:09:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~210s 2024-05-09T21:09:47.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~210s 2024-05-09T21:09:47.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~210s 2024-05-09T21:09:47.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~210s 2024-05-09T21:09:47.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~210s 2024-05-09T21:09:47.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~210s 2024-05-09T21:09:52.253 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~215s 2024-05-09T21:09:52.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~215s 2024-05-09T21:09:52.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~215s 2024-05-09T21:09:52.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~215s 2024-05-09T21:09:52.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~215s 2024-05-09T21:09:52.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~215s 2024-05-09T21:09:52.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~215s 2024-05-09T21:09:52.254 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~215s 2024-05-09T21:09:52.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~215s 2024-05-09T21:09:52.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~215s 2024-05-09T21:09:52.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~215s 2024-05-09T21:09:52.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~215s 2024-05-09T21:09:52.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~215s 2024-05-09T21:09:52.255 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~215s 2024-05-09T21:09:57.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~220s 2024-05-09T21:09:57.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~220s 2024-05-09T21:09:57.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~220s 2024-05-09T21:09:57.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~220s 2024-05-09T21:09:57.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~220s 2024-05-09T21:09:57.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~220s 2024-05-09T21:09:57.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~220s 2024-05-09T21:09:57.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~220s 2024-05-09T21:09:57.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~220s 2024-05-09T21:09:57.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~220s 2024-05-09T21:09:57.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~220s 2024-05-09T21:09:57.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~220s 2024-05-09T21:09:57.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~220s 2024-05-09T21:09:57.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~220s 2024-05-09T21:10:02.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~225s 2024-05-09T21:10:02.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~225s 2024-05-09T21:10:02.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~225s 2024-05-09T21:10:02.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~225s 2024-05-09T21:10:02.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~225s 2024-05-09T21:10:02.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~225s 2024-05-09T21:10:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~225s 2024-05-09T21:10:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~225s 2024-05-09T21:10:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~225s 2024-05-09T21:10:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~225s 2024-05-09T21:10:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~225s 2024-05-09T21:10:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~225s 2024-05-09T21:10:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~225s 2024-05-09T21:10:02.261 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~225s 2024-05-09T21:10:07.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~230s 2024-05-09T21:10:07.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~230s 2024-05-09T21:10:07.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~230s 2024-05-09T21:10:07.263 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~230s 2024-05-09T21:10:07.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~230s 2024-05-09T21:10:07.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~230s 2024-05-09T21:10:12.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~235s 2024-05-09T21:10:12.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~235s 2024-05-09T21:10:12.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~235s 2024-05-09T21:10:12.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~235s 2024-05-09T21:10:12.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~235s 2024-05-09T21:10:12.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~235s 2024-05-09T21:10:12.266 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~235s 2024-05-09T21:10:12.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~235s 2024-05-09T21:10:12.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~235s 2024-05-09T21:10:12.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~235s 2024-05-09T21:10:12.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~235s 2024-05-09T21:10:12.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~235s 2024-05-09T21:10:12.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~235s 2024-05-09T21:10:12.267 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~235s 2024-05-09T21:10:17.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~240s 2024-05-09T21:10:17.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~240s 2024-05-09T21:10:17.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~240s 2024-05-09T21:10:17.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~240s 2024-05-09T21:10:17.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~240s 2024-05-09T21:10:17.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~240s 2024-05-09T21:10:17.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~240s 2024-05-09T21:10:17.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~240s 2024-05-09T21:10:17.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~240s 2024-05-09T21:10:17.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~240s 2024-05-09T21:10:17.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~240s 2024-05-09T21:10:17.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~240s 2024-05-09T21:10:17.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~240s 2024-05-09T21:10:17.270 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~240s 2024-05-09T21:10:22.271 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~245s 2024-05-09T21:10:22.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~245s 2024-05-09T21:10:22.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~245s 2024-05-09T21:10:22.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~245s 2024-05-09T21:10:22.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~245s 2024-05-09T21:10:22.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~245s 2024-05-09T21:10:22.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~245s 2024-05-09T21:10:22.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~245s 2024-05-09T21:10:22.272 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~245s 2024-05-09T21:10:22.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~245s 2024-05-09T21:10:22.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~245s 2024-05-09T21:10:22.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~245s 2024-05-09T21:10:22.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~245s 2024-05-09T21:10:22.273 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~245s 2024-05-09T21:10:27.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~250s 2024-05-09T21:10:27.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~250s 2024-05-09T21:10:27.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~250s 2024-05-09T21:10:27.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~250s 2024-05-09T21:10:27.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~250s 2024-05-09T21:10:27.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~250s 2024-05-09T21:10:32.278 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~255s 2024-05-09T21:10:32.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~255s 2024-05-09T21:10:32.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~255s 2024-05-09T21:10:32.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~255s 2024-05-09T21:10:32.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~255s 2024-05-09T21:10:32.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~255s 2024-05-09T21:10:37.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~260s 2024-05-09T21:10:37.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~260s 2024-05-09T21:10:37.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~260s 2024-05-09T21:10:37.281 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~260s 2024-05-09T21:10:37.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~260s 2024-05-09T21:10:37.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~260s 2024-05-09T21:10:37.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~260s 2024-05-09T21:10:37.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~260s 2024-05-09T21:10:37.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~260s 2024-05-09T21:10:37.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~260s 2024-05-09T21:10:37.282 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~260s 2024-05-09T21:10:37.283 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~260s 2024-05-09T21:10:37.283 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~260s 2024-05-09T21:10:37.283 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~260s 2024-05-09T21:10:42.284 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~265s 2024-05-09T21:10:42.285 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~265s 2024-05-09T21:10:42.286 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~265s 2024-05-09T21:10:42.286 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~265s 2024-05-09T21:10:42.286 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~265s 2024-05-09T21:10:42.286 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~265s 2024-05-09T21:10:47.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~270s 2024-05-09T21:10:47.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~270s 2024-05-09T21:10:47.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~270s 2024-05-09T21:10:47.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~270s 2024-05-09T21:10:47.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~270s 2024-05-09T21:10:47.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~270s 2024-05-09T21:10:47.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~270s 2024-05-09T21:10:47.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~270s 2024-05-09T21:10:47.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~270s 2024-05-09T21:10:47.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~270s 2024-05-09T21:10:47.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~270s 2024-05-09T21:10:47.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~270s 2024-05-09T21:10:47.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~270s 2024-05-09T21:10:47.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~270s 2024-05-09T21:10:52.292 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~275s 2024-05-09T21:10:52.293 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~275s 2024-05-09T21:10:52.294 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~275s 2024-05-09T21:10:52.294 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~275s 2024-05-09T21:10:52.294 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~275s 2024-05-09T21:10:57.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~280s 2024-05-09T21:10:57.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~280s 2024-05-09T21:10:57.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~280s 2024-05-09T21:10:57.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~280s 2024-05-09T21:10:57.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~280s 2024-05-09T21:10:57.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~280s 2024-05-09T21:10:57.295 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~280s 2024-05-09T21:10:57.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~280s 2024-05-09T21:10:57.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~280s 2024-05-09T21:10:57.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~280s 2024-05-09T21:10:57.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~280s 2024-05-09T21:10:57.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~280s 2024-05-09T21:10:57.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~280s 2024-05-09T21:10:57.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~280s 2024-05-09T21:11:02.297 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~285s 2024-05-09T21:11:02.297 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~285s 2024-05-09T21:11:02.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~285s 2024-05-09T21:11:02.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~285s 2024-05-09T21:11:02.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~285s 2024-05-09T21:11:02.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~285s 2024-05-09T21:11:02.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~285s 2024-05-09T21:11:02.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~285s 2024-05-09T21:11:02.298 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~285s 2024-05-09T21:11:02.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~285s 2024-05-09T21:11:02.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~285s 2024-05-09T21:11:02.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~285s 2024-05-09T21:11:02.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~285s 2024-05-09T21:11:02.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~285s 2024-05-09T21:11:07.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~290s 2024-05-09T21:11:07.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~290s 2024-05-09T21:11:07.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~290s 2024-05-09T21:11:07.301 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~290s 2024-05-09T21:11:07.302 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~290s 2024-05-09T21:11:12.302 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~295s 2024-05-09T21:11:12.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~295s 2024-05-09T21:11:12.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~295s 2024-05-09T21:11:12.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~295s 2024-05-09T21:11:12.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~295s 2024-05-09T21:11:12.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~295s 2024-05-09T21:11:12.303 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~295s 2024-05-09T21:11:12.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~295s 2024-05-09T21:11:12.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~295s 2024-05-09T21:11:12.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~295s 2024-05-09T21:11:12.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~295s 2024-05-09T21:11:12.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~295s 2024-05-09T21:11:12.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~295s 2024-05-09T21:11:12.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~295s 2024-05-09T21:11:17.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~300s 2024-05-09T21:11:17.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~300s 2024-05-09T21:11:17.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~300s 2024-05-09T21:11:17.308 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~300s 2024-05-09T21:11:17.308 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~300s 2024-05-09T21:11:17.308 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~300s 2024-05-09T21:11:17.308 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2024-05-10T07:56:35.988 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-05-10T07:56:36.009 DEBUG:teuthology.task.console_log:Killing console logger for smithi063 2024-05-10T07:56:36.010 DEBUG:teuthology.task.console_log:Killing console logger for smithi120 2024-05-10T07:56:36.010 DEBUG:teuthology.exit:Finished running handlers